Skip to main content

​
Synopsis

ramalama convert [options] model [target]

​
Description

Convert specified AI Model to an OCI Formatted AI Model The model can be from RamaLama model storage in Huggingface, Ollama, or a local model stored on disk. Converting from an OCI model is not supported.
The convert command must be run with containers. Use of the —nocontainer option is not allowed.

​
Options

​
—gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0

Convert Safetensor models into a GGUF with the specified quantization format. To learn more about model quantization, read llama.cpp documentation: https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md

​
—help, -h

Print usage message

​
—network=none

sets the configuration for network namespaces when handling RUN instructions

​
—type=raw | car

type of OCI Model Image to convert.
TypeDescription
carIncludes base image with the model stored in a /models subdir
rawOnly the model and a link file model.file to it stored at /

​
EXAMPLE

Generate an oci model out of an Ollama model. 
$ ramalama convert ollama://tinyllama:latest oci://quay.io/rhatdan/tiny:latest
Building quay.io/rhatdan/tiny:latest...
STEP 1/2: FROM scratch
STEP 2/2: COPY sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 /model
--> Using cache 69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
COMMIT quay.io/rhatdan/tiny:latest
--> 69db4a10191c
Successfully tagged quay.io/rhatdan/tiny:latest
69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
Generate and run an oci model with a quantized GGUF converted from Safetensors. 
$ ramalama --image quay.io/ramalama/ramalama-rag convert --gguf Q4_K_M hf://ibm-granite/granite-3.2-2b-instruct oci://quay.io/kugupta/granite-3.2-q4-k-m:latest
Converting /Users/kugupta/.local/share/ramalama/models/huggingface/ibm-granite/granite-3.2-2b-instruct to quay.io/kugupta/granite-3.2-q4-k-m:latest...
Building quay.io/kugupta/granite-3.2-q4-k-m:latest...
$ ramalama run oci://quay.io/kugupta/granite-3.2-q4-k-m:latest

​
See Also

ramalama(1), ramalama-push(1)
Aug 2024, Originally compiled by Eric Curtin <ecurtin@redhat.com>