Modello come artefatto OCI

Pacchetto del modello (lato connesso)

Formato Ollama (per Ollama in-cluster):

ollama pull qwen2.5:14b-instruct

M=models/manifests/registry.ollama.ai/library/qwen2.5/14b-instruct

tar -C ~/.ollama -czf qwen2.5-14b-instruct-ollama.tgz $M $(cd ~/.ollama && jq -r '.layers[].digest, .config.digest' $M | sed 's/sha256:/models\/blobs\/sha256-/')

shasum -a 256 qwen2.5-14b-instruct-ollama.tgz > qwen2.5-14b-instruct-ollama.tgz.sha256

Formato Hugging Face (per vLLM, variante AWQ adatta a GPU da 24 GB):

huggingface-cli download Qwen/Qwen2.5-14B-Instruct-AWQ --local-dir ./qwen2.5-14b-instruct-awq

tar -czf qwen2.5-14b-instruct-awq.tgz qwen2.5-14b-instruct-awq

shasum -a 256 qwen2.5-14b-instruct-awq.tgz > qwen2.5-14b-instruct-awq.tgz.sha256

Push nel registry

oras login registry.internal

oras push registry.internal/ai-models/qwen2.5-14b-instruct:ollama-q4_k_m qwen2.5-14b-instruct-ollama.tgz:application/gzip qwen2.5-14b-instruct-ollama.tgz.sha256:text/plain --annotation org.opencontainers.image.source=ollama://qwen2.5:14b-instruct

oras push registry.internal/ai-models/qwen2.5-14b-instruct:hf-awq qwen2.5-14b-instruct-awq.tgz:application/gzip qwen2.5-14b-instruct-awq.tgz.sha256:text/plain --annotation org.opencontainers.image.source=https://huggingface.co/Qwen/Qwen2.5-14B-Instruct-AWQ

oras resolve registry.internal/ai-models/qwen2.5-14b-instruct:ollama-q4_k_m

Immagine model-fetcher

Un’immagine minima con oras, tar e sha256sum, usata come initContainer. Build lato connesso:

mkdir -p model-fetcher && cd model-fetcher

cat > Containerfile <<'EOF2'
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
ARG ORAS_VERSION=1.2.0
RUN microdnf install -y tar gzip findutils && microdnf clean all && \
    curl -sSL https://github.com/oras-project/oras/releases/download/v${ORAS_VERSION}/oras_${ORAS_VERSION}_linux_amd64.tar.gz \
    | tar -xz -C /usr/local/bin oras
COPY fetch-model.sh /usr/local/bin/fetch-model.sh
RUN chmod 0755 /usr/local/bin/fetch-model.sh
USER 1001
ENTRYPOINT ["/usr/local/bin/fetch-model.sh"]
EOF2

cat > fetch-model.sh <<'EOF2'
#!/bin/sh
# Env: MODEL_REF (oci ref), DEST (target dir), optional REGISTRY_CONFIG, CA_FILE
set -eu
MARKER="$DEST/.model-$(echo "$MODEL_REF" | sha256sum | cut -c1-12)"
if [ -f "$MARKER" ]; then echo "model already present: $MODEL_REF"; exit 0; fi
STAGE=$(mktemp -d)
OPTS=""
[ -n "${REGISTRY_CONFIG:-}" ] && OPTS="$OPTS --registry-config $REGISTRY_CONFIG"
[ -n "${CA_FILE:-}" ] && OPTS="$OPTS --ca-file $CA_FILE"
oras pull $OPTS "$MODEL_REF" -o "$STAGE"
cd "$STAGE" && sha256sum -c ./*.sha256
mkdir -p "$DEST" && tar -xzf ./*.tgz -C "$DEST"
touch "$MARKER"
echo "model ready in $DEST"
EOF2

podman build --platform linux/amd64 -t registry.internal/ai/model-fetcher:1.0 .

podman push registry.internal/ai/model-fetcher:1.0

Il marker evita di riscaricare il modello a ogni restart; cambiando MODEL_REF il nuovo modello viene scaricato accanto al precedente.

Credenziali del registry nel cluster

kubectl create namespace ai-llm

kubectl -n ai-llm create secret docker-registry registry-internal --docker-server=registry.internal --docker-username=<user> --docker-password=<token>

kubectl -n ai-llm create configmap registry-ca --from-file=ca.crt=/path/to/registry-ca.crt