> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ramalama.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Quick start

> Run your first local model with the Python SDK.

The SDK spins up a local model server and lets you chat with it using a simple API.

## Run a model

### Context Managers

The context manager will automatically manage and clean up running models on your behalf.

```python theme={"system"}
from ramalama_sdk import RamalamaModel

with RamalamaModel(model="tinyllama") as model:
    response = model.chat("How tall is Michael Jordan?")
    print(response["content"])
```

### Manual Management

It's also possible to manually manage the models run state.

```python title="Manual lifecycle" theme={"system"}
from ramalama_sdk import RamalamaModel

model_name = "tinyllama"
model = RamalamaModel(model=model_name)
model.download()
model.serve()
```

Once the model is serving, you can call the local OpenAI-compatible endpoint yourself.

<CodeGroup>
  ```python title="SDK chat" theme={"system"}
  try:
      response = model.chat("How tall is Michael Jordan?")
      print(response["content"])
  finally:
      model.stop()
  ```

  ```python title="Requests" theme={"system"}
  import requests

  model_name = "tinyllama"
  url = f"{model.server_attributes.url}/chat/completions"
  payload = {
      "model": model_name,
      "messages": [
          {"role": "user", "content": "Write a short limerick about llamas."}
      ]
  }

  response = requests.post(url, json=payload, timeout=60)
  print(response.json())
  ```
</CodeGroup>

## Download models

Use `download()` to fetch and cache models before serving.
The model identifier controls where the SDK pulls from.

Common prefixes include

* HuggingFace: `hf://`
* Ollama: `ollama://`
* OCI (any oci image repository): `oci://`
* ModelScope: `modelscope://`
* File: `file://`

<CodeGroup>
  ```python title="HuggingFace" theme={"system"}
  from ramalama_sdk import RamalamaModel

  model = RamalamaModel(model="hf://ggml-org/gpt-oss-20b-GGUF")
  model.download()
  ```

  ```python title="Ollama" theme={"system"}
  from ramalama_sdk import RamalamaModel

  model = RamalamaModel(model="ollama://deepseek-r1")
  model.download()
  ```

  ```python title="OCI" theme={"system"}
  from ramalama_sdk import RamalamaModel

  model = RamalamaModel(model="oci://rlcr.io/ramalama/smollm3-3b:latest")
  model.download()
  ```

  ```python title="Local file" theme={"system"}
  from ramalama_sdk import RamalamaModel

  model = RamalamaModel(model="file://<your-local-model>.gguf")
  model.download()
  ```
</CodeGroup>

## Instantiating a model

You can pass runtime overrides when creating a model session:

```python theme={"system"}
from ramalama_sdk import RamalamaModel

model = RamalamaModel(
    model="tinyllama",
    base_image=None,
    temp=0.7,
    ngl=20,
    max_tokens=256,
    threads=8,
    ctx_size=4096,
    timeout=30,
)
```

| Parameter   | Type          | Description                                                   | Default  |
| ----------- | ------------- | ------------------------------------------------------------- | -------- |
| model       | str           | Model name or identifier.                                     | required |
| base\_image | str or None   | Container image to use for serving, if different from config. | None     |
| temp        | float or None | Temperature override for sampling.                            | None     |
| ngl         | int or None   | GPU layers override.                                          | None     |
| max\_tokens | int or None   | Maximum tokens for completions.                               | None     |
| threads     | int or None   | CPU threads override.                                         | None     |
| ctx\_size   | int or None   | Context window override.                                      | None     |
| timeout     | int           | Seconds to wait for server readiness.                         | 30       |