Models (OCI)

RamaLama “Model” artifacts package raw model files (e.g., .gguf, Safetensors) using the OCI format. They are registry‑hosted, content‑addressed, and provenance‑rich — ideal for reproducible deployments, enterprise controls, and air‑gapped environments.

Why use OCI‑packaged models

Portability: Pull the same model to any node that can reach your registry
Provenance: Standardized annotations for origin, license, and file metadata
Separation of concerns: Update models independently of runtimes and apps
Air‑gapped: Mirror/pull once, distribute internally, mount read‑only

Tags and discovery

Use content tags like :gguf when pulling GGUF model files
“Image‑as‑volume” variants use :gguf-image (for Podman --mount type=image)
Browse tags: https://registry.ramalama.com/projects/ramalama
Pull artifacts from: rlcr.io/ramalama/...

Pull models locally

Use a tool like ORAS to download model files to disk, or reference the artifact directly with the RamaLama CLI.

oras pull rlcr.io/ramalama/gemma3-270m:gguf -o ./models/

ramalama serve --image rlcr.io/ramalama/llamacpp-cpu-distroless oci://rlcr.io/ramalama/gemma3-270m:gguf

You can find the full catalogue of RamaLama Labs images here

Run with a runtime

Mount the model directory into a runtime container and pass the path to --model.

docker run --rm -p 8080:8080 \
  -v "$PWD/models:/models:ro" \
  rlcr.io/ramalama/llamacpp-cpu-distroless:latest \
  --model /models/gemma-3-270m-it-Q6_K.gguf --host 0.0.0.0 --port 8080

docker run --rm -p 8080:8080 --gpus all \
  -v "$PWD/models:/models:ro" \
  rlcr.io/ramalama/llamacpp-cuda-distroless:latest \
  --model /models/gemma-3-270m-it-Q6_K.gguf --host 0.0.0.0 --port 8080

podman run --rm -p 8080:8080 \
  -v "$PWD/models:/models:ro" \
  rlcr.io/ramalama/llamacpp-cpu-distroless:latest \
  --model /models/gemma-3-270m-it-Q6_K.gguf --host 0.0.0.0 --port 8080

Podman: Image‑as‑volume

Avoid a local models directory by mounting the OCI model artifact as a read‑only image volume.

Podman

podman run --rm -p 8080:8080 \
  --mount type=image,src=rlcr.io/ramalama/gemma3-270m:gguf-image,target=/artifact,ro=true \
  rlcr.io/ramalama/llamacpp-cpu-distroless:latest \
  --model /artifact/models/<exact-file>.gguf --host 0.0.0.0 --port 8080

Need the exact model filename? Inspect labels/annotations attached to artifacts. See the examples in /pages/deploying/compose under “Other Notes”.

Getting Started

Quick Start

Deploying

Artifact Types

Education

Why use OCI‑packaged models

Tags and discovery

Pull models locally

Run with a runtime

Podman: Image‑as‑volume

See also

​Why use OCI‑packaged models

​Tags and discovery

​Pull models locally

​Run with a runtime

​Podman: Image‑as‑volume

​See also

Why use OCI‑packaged models

Tags and discovery

Pull models locally

Run with a runtime

Podman: Image‑as‑volume

See also