Model name for Python API endpoint

[hkcowboy]

Hello, I am trying to use the DeepSeek R1 llamafile as the back end for Perplexica, but have a question: how do I find the model name to provide?

Right now my API info is:
API_KEY="something"
API_URL="http://localhost:8080/v1"
MODEL_NAME="DeepSeek-R1-Distill-Llama-8B-Q4_K_M" (not sure if this is correct)

But the model name seems incorrect - thank you in advance.

[KeepALifeUS]

Model name configuration for Python API endpoints is important for compatibility. Here is a pattern:

Model Name Mapping

from typing import Optional
import os

class LlamafileClient:
    def __init__(
        self,
        base_url: str = "http://localhost:8080",
        model_name: Optional[str] = None
    ):
        self.base_url = base_url
        # Allow explicit model name or derive from llamafile
        self.model_name = model_name or self._detect_model_name()
    
    def _detect_model_name(self) -> str:
        """Try to detect model name from llamafile metadata"""
        try:
            # Option 1: Query /v1/models endpoint
            import requests
            resp = requests.get(f"{self.base_url}/v1/models")
            if resp.ok:
                models = resp.json().get("data", [])
                if models:
                    return models[0]["id"]
        except:
            pass
        
        # Option 2: Use environment variable
        return os.getenv("LLAMAFILE_MODEL_NAME", "llamafile-local")
    
    def create_completion(self, messages: list, **kwargs):
        import requests
        
        return requests.post(
            f"{self.base_url}/v1/chat/completions",
            json={
                "model": self.model_name,  # Use configured name
                "messages": messages,
                **kwargs
            }
        ).json()

OpenAI SDK Compatibility

from openai import OpenAI

# Set model name explicitly
client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed"
)

# Use any model name - llamafile ignores it but some tools need it
response = client.chat.completions.create(
    model="llama-3-8b",  # Name for logging/tracking purposes
    messages=[{"role": "user", "content": "Hello"}]
)

Best Practice

Start llamafile with explicit model name:

./model.llamafile --server --model-name "llama-3-8b-instruct"

This makes the /v1/models endpoint return meaningful data.

More patterns: https://github.com/KeepALifeUS/autonomous-agents