Runner catalog¶
The runner catalog lets operators browse a curated list of engine runners and import them (pull their SIF onto the shared module store) from the dashboard — no need to build SIFs yourself.
After importing MLServer, follow Deploying models with MLServer for supported
formats, model directory layout, and model-settings.json examples.
How it works¶
- Official runners are built + signed, then pushed to an OCI registry as ORAS artifacts
(
apptainer push my.sif oras://quay.io/rh-aiservices-bu/sardeenz-runners/<engine>:<tag>). - A catalog file (
runners.yaml) lists the available runners (title, description, engine, version, ORAS image,sifName, tags,protocol,entrypoint, …).protocol(required:openai|oip) names the proxy protocol family the runner's models are invoked under —openaifor OpenAI-compatible engines (vLLM),oipfor KServe V2 Open Inference Protocol engines (MLServer).entrypoint(optional argv) is the verbatim command the worker execs inside the SIF to launch the runner's management shim, falling back to the worker'sSARDEENZ_RUNNER_ENTRYPOINTwhen absent. See ADR-021. The default catalog is the officialschool-of-sardeenz/runners.yaml; pointSARDEENZ_RUNNER_CATALOG_URLat your own to customize. The repo-rootrunners.yamlis the dev source and the schema reference. - The control plane loads the catalog, cross-references the module store, and serves a merged view (which entries are imported, whether an update is available, and any module-store SIFs not in the catalog). Manual Refresh bypasses HTTP and intermediary caches so a mutable catalog URL is re-read from its origin. The dashboard shows when that fetch completed and also refreshes live via the event stream during imports.
- Import resolves the digest-pinned OCI manifest, streams its single SIF layer directly onto
the module store with byte-level progress, verifies the layer digest and SIF signature, and
publishes it atomically. The imported image digest is recorded in a
<sifName>.sif.metadata.jsonsidecar. If the catalog later advertises a different digest for the same engine version, the dashboard marks an update available; Re-import installs it. Legacy SIFs without sidecar metadata are conservatively marked as having an update until re-imported once. Uninstall deletes both files (blocked while a runner started from the SIF is running). The librarian build pipeline remains for building your own SIFs.
Configuration¶
| Var | Purpose | Default |
|---|---|---|
SARDEENZ_RUNNER_CATALOG_URL |
Catalog source — http(s) URL, local path, or file:// |
official school-of-sardeenz raw URL |
SARDEENZ_ALLOW_INSECURE_CATALOG |
Allow an http:// (plaintext) catalog source. Off by default — a plaintext catalog can be rewritten in transit. Prefer https:// or a local path/file:// instead of enabling this |
false |
SARDEENZ_MODULES_DIR |
Shared module store path | /modules |
SARDEENZ_SIF_IMPORTER |
oras (real OCI SIF stream + verification) or stub (explicit dev/test placeholder that cannot execute) |
oras |
SARDEENZ_VERIFY_SIF |
apptainer verify SIFs — at catalog import (control plane) and at exec (worker). Set false for unsigned experimentation only (see below) |
true |
APPTAINER_AUTH_FILE |
Docker-format registry credential file used by OCI imports; credentials are sent only after an HTTPS registry authentication challenge | unset (public registries only) |
ORAS image references (image: oras://...) in the catalog must be digest-pinned
(oras://<registry>/<repo>:<tag>@sha256:<digest>). A mutable tag can be repointed after a catalog
entry was reviewed, silently changing what gets pulled onto the module store; entries without a
digest are skipped (logged) rather than imported. scripts/build-sif.sh enforces the same
digest-pinning requirement on its --image input.
Deployment requirements (RW module mount, sardeenz-control-plane SA, apptainer in the image,
signing public key) are in deployment/control-plane/.
Publishing an official runner (maintainers)¶
- Build + sign the runner SIF (see
containers/runners/vllm/0.21.0and the librarian pipeline, orapptainer build+apptainer sign). - Push it via ORAS:
apptainer push <engine>-<version>.sif oras://quay.io/rh-aiservices-bu/sardeenz-runners/<engine>:<tag>. - Add an entry to the catalog
runners.yaml(schema in the repo-root sample).sifNameby convention follows the<engine>-<version>shape and must match^[A-Za-z0-9_.-]+$(it becomes theruntimeModulea worker execs). - Sign SIFs with the key whose public half is distributed to workers and the control plane, so
apptainer verifytrusts them at import and exec.
Unsigned SIFs (experimentation only)¶
Signing/verification is defense-in-depth for a shared cluster, not a prerequisite for getting runners working. During early experimentation — no CI, no librarian, no signing key yet — you can skip it entirely and turn it back on later (it is a runtime toggle; nothing built now is wasted).
- Use the parameterized librarian pipeline with
SIGN_SIF=false, or build and push manually withoutapptainer sign:
export APPTAINER_TMPDIR=/scratch APPTAINER_CACHEDIR=/scratch/cache
apptainer build vllm-0.21.sif docker://quay.io/rh-aiservices-bu/sardeenz-runner-images/vllm:0.21
apptainer push --allow-unsigned vllm-0.21.sif \
oras://quay.io/rh-aiservices-bu/sardeenz-runners/vllm:0.21
- Set
SARDEENZ_VERIFY_SIF=falseon both the control plane (skips verify after the OCI download) and the workers (skips verify before exec). Locally that is one line in.env, which drives both; on a cluster set the env var on the control-plane and worker Deployments.
Anyone with registry/store write access can then build and publish — no keys, no coordination.
Security trade-off. Access control answers who may write; the signature answers is this SIF byte-for-byte what was built. With verification off, anything that gains write access (leaked credentials, a bad job, a half-written file) can run code on your GPUs unquestioned. SIFs on an RWX volume / ORAS artifacts also bypass the cluster's normal image-admission checks. Keep an unsigned instance inside a trusted sandbox, and re-enable
SARDEENZ_VERIFY_SIFwith a shared signing key (private half in CI/librarian, public half distributed to workers + the control plane — seedeployment/librarian/) before exposing or sharing the deployment.
Example: an MLServer (OIP) entry¶
This catalog entry installs the runtime. Preparing model artifacts and selecting the correct file are covered in the MLServer model deployment guide.
- id: mlserver-1.7
title: MLServer 1.7 (KServe V2)
description: Seldon MLServer 1.7 — KServe V2 Open Inference Protocol, sklearn/HF/ONNX-backed.
engine: MLServer
runnerType: mlserver
version: '1.7'
image: oras://quay.io/rh-aiservices-bu/sardeenz-runners/mlserver:1.7@sha256:<digest>
sifName: mlserver-1.7
protocol: oip
entrypoint: [python3, -m, sardeenz_mlserver_runner]
supportedModelTypes: [PREDICTIVE, LLM, EMBEDDING]
Authoring your own catalog¶
Copy runners.yaml, host it anywhere reachable (an internal URL, a Git raw URL, or a file mounted
into the control plane), and set SARDEENZ_RUNNER_CATALOG_URL.
Entries that fail validation are skipped, but a missing or invalid protocol is now a loud,
surfaced validation error — such entries are excluded from the imported list, reported in
RunnerCatalogView.invalidEntries, and shown in the dashboard's "some catalog entries were
skipped" warning, rather than silently dropped. This keeps an out-of-date or malformed catalog from
presenting as "no runners available" with no explanation.
Importing a runner whose protocol the running proxy does not advertise fails at import time
with an actionable "proxy upgrade required" (409) error — the proxy publishes its supported
protocol set at the {prefix}:proxy:protocols Redis key on every (re)connect, and catalog import
checks it as a forward-compat guard (fail-open when the key is absent, e.g. the proxy hasn't
started yet). See ADR-021.