convert

Synopsis

ramalama convert [options] model [target]

Description

Convert specified AI Model to an OCI Formatted AI Model The model can be from RamaLama model storage in Huggingface, Ollama, or a local model stored on disk. Converting from an OCI model is not supported.

The convert command must be run with containers. Use of the —nocontainer option is not allowed.

Options

—gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0

Convert Safetensor models into a GGUF with the specified quantization format. To learn more about model quantization, read llama.cpp documentation: https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md

—help, -h

Print usage message

—network=none

sets the configuration for network namespaces when handling RUN instructions

—type=raw | car

type of OCI Model Image to convert.

Type	Description
car	Includes base image with the model stored in a /models subdir
raw	Only the model and a link file model.file to it stored at /

EXAMPLE

Generate an oci model out of an Ollama model.

$ ramalama convert ollama://tinyllama:latest oci://quay.io/rhatdan/tiny:latest
Building quay.io/rhatdan/tiny:latest...
STEP 1/2: FROM scratch
STEP 2/2: COPY sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 /model
--> Using cache 69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
COMMIT quay.io/rhatdan/tiny:latest
--> 69db4a10191c
Successfully tagged quay.io/rhatdan/tiny:latest
69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344

Generate and run an oci model with a quantized GGUF converted from Safetensors.

$ ramalama --image quay.io/ramalama/ramalama-rag convert --gguf Q4_K_M hf://ibm-granite/granite-3.2-2b-instruct oci://quay.io/kugupta/granite-3.2-q4-k-m:latest
Converting /Users/kugupta/.local/share/ramalama/models/huggingface/ibm-granite/granite-3.2-2b-instruct to quay.io/kugupta/granite-3.2-q4-k-m:latest...
Building quay.io/kugupta/granite-3.2-q4-k-m:latest...
$ ramalama run oci://quay.io/kugupta/granite-3.2-q4-k-m:latest

Overview

Configuration

Platform Guides

Commands

Synopsis

Description

Options

—gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0

—help, -h

—network=none

—type=raw | car

EXAMPLE

See Also

​Synopsis

​Description

​Options

​—gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0

​—help, -h

​—network=none

​—type=raw | car

​EXAMPLE

​See Also

Synopsis

Description

Options

—gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0

—help, -h

—network=none

—type=raw | car

EXAMPLE

See Also