Model Images - Ramalama Labs Docs

Model images package both an inference runtime (e.g., llama.cpp) and a specific model into a single container image. They’re ideal for quick starts, demos, single‑purpose services, and environments where simplicity is preferred over component isolation.

When to use model images

Fastest way to get an endpoint running
Minimal choices: no need to choose a runtime or mount model files
Great for laptops, POCs, and small dedicated services

If you need stronger isolation or to manage model files independently, see /pages/artifacts/runtime and /pages/artifacts/model.

Quick start

docker pull rlcr.io/ramalama/gemma3-270m:latest
docker run --rm -p 8080:8080 rlcr.io/ramalama/gemma3-270m:latest

podman pull rlcr.io/ramalama/gemma3-270m:latest
podman run --rm -p 8080:8080 rlcr.io/ramalama/gemma3-270m:latest

ramalama serve --image rlcr.io/ramalama/llamacpp-cpu-distroless rlcr://gemma3-270m

Test the OpenAI‑compatible API:

curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"gemma3-270m","messages":[{"role":"user","content":"Say hello in one sentence"}]}'

ramalama chat "Say hello in one sentence"

You can find the full catalogue of RamaLama Labs images here

Compose

docker-compose.yaml

services:
  ai:
    image: rlcr.io/ramalama/gemma3-270m:latest
    ports:
      - "8080:8080"
    restart: unless-stopped

Notes on updates, tags, and hardware

Examples use :latest; pin tags in production for repeatability
Images are rebuilt and scanned regularly for security and performance
Hardware acceleration is chosen by the underlying image; for advanced control, use runtimes directly

See also

Manage models separately: /pages/artifacts/model
Engines only (mount a model): /pages/artifacts/runtime