models

package
v0.4.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 30, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package models abstracts model providers and client interfaces.

Registry resolves namespace/model_id strings using Project.spec.providers.models. Use MockClient for deterministic tests; OpenAIClient is the MVP OpenAI-compatible backend (§12.2 F). MockClient.Script drives a tool-calling loop without a live provider: each Generate consumes the next MockTurn (for example turn 1 → StopReasonToolUse, turn 2 → final text/JSON). Requests, including GenerateRequest.Tools, are recorded for assertions; per-call token counts go on MockTurn.Meta. When tokens are set and CostUSD is 0, CostUSD is filled from the same B1 price table as live adapters (issue #164); an explicit non-zero CostUSD is kept. Without Script, MockClient still returns a fixed MockClient.Content, except when GenerateRequest.Tools is non-empty: a restart-like tool name drives the #167 tool_use then status JSON; otherwise the first advertised non-restart tool is called once and the next Generate returns Content (issue #168). Agents with no tools stay single-shot Content. Each empty-Script Generate uses MockClient.Meta (Registry mock sets CostUSD 0.02).

Model contract (issue #156)

GenerateRequest carries chat ChatMessage turns plus optional ToolDef definitions. GenerateRequest.ToolChoice controls whether the model may call tools: "auto" (default), "none", or "required". The zero value of ToolChoice behaves as "auto" via GenerateRequest.ToolChoiceOrDefault.

GenerateResponse returns assistant GenerateResponse.Content, optional ToolCall requests when GenerateResponse.StopReason is StopReasonToolUse, and GenerateMeta accounting. Other known stop reasons include StopReasonEndTurn and StopReasonMaxTokens. Unknown provider finish reasons are passed through; treat them as non-success, not as end_turn.

Tool results are returned to the model on ChatMessage.ToolResults (not a separate message role). Each ToolResult references the originating ToolCall.ID in ToolResult.ToolCallID. Replay the assistant turn that requested tools on ChatMessage.ToolCalls so the next Generate can include native tool-call blocks before the results.

OpenAIClient maps this contract to Chat Completions `tools` / `tool_calls` / `role: "tool"` (issue #157). It clears GenerateResponse.ToolCalls unless GenerateResponse.StopReason is StopReasonToolUse (for example `length` or `content_filter`).

xAI Grok, Google Gemini, and Moonshot Kimi expose OpenAI-compatible Chat Completions endpoints, so they reuse OpenAIClient with only the base URL and pricing table changed — see NewGrokClientFromConfig (provider type "grok"), NewGeminiClientFromConfig ("gemini"), and NewKimiClientFromConfig ("kimi"). OpenAIClient.CostProvider selects the pricing table so CostUSD is estimated from that provider's rows in tokenUSDPerMillion; unknown model ids stay at 0.

The Anthropic adapter maps the same contract to Messages API `tools` / `tool_use` / `tool_result` (issue #158). `ToolChoiceRequired` becomes `tool_choice.type=any`. Follow-up `ToolResults` are user `tool_result` blocks (Anthropic has no `role: "tool"` and requires user/assistant alternation, so extra ChatMessage.Content on a result turn stays in that same user message, and consecutive ToolResults ChatMessages are merged into one user turn). Empty tool output still sends `content: ""`. Token usage comes from `usage.input_tokens` / `usage.output_tokens`. GenerateMeta.CostUSD is estimated from those counts × the Anthropic rows in tokenUSDPerMillion (issue #162); unknown model ids stay at 0. Plain `end_turn` responses with no text still error; `tool_use` may omit text.

Index

Constants

View Source
const (
	ToolChoiceAuto     = "auto"
	ToolChoiceNone     = "none"
	ToolChoiceRequired = "required"
)

Tool choice constants for GenerateRequest.ToolChoice. The zero value of ToolChoice behaves as ToolChoiceAuto.

View Source
const (
	StopReasonEndTurn   = "end_turn"
	StopReasonToolUse   = "tool_use"
	StopReasonMaxTokens = "max_tokens"
)

Stop reason constants for GenerateResponse.StopReason.

Variables

This section is empty.

Functions

func BuiltinProviderConfig added in v0.4.0

func BuiltinProviderConfig(ns string) (spec.ModelProviderConfig, bool)

BuiltinProviderConfig returns the config a built-in namespace resolves to implicitly, or false when ns is not built-in. Exposed so tooling (e.g. `terfyn migrate`) can drop a providers.models entry that merely restates a built-in namespace, which needs no declaration in .agent (issue #440).

func NewAnthropicClientFromConfig

func NewAnthropicClientFromConfig(cfg spec.ModelProviderConfig) (*anthropicClient, error)

NewAnthropicClientFromConfig builds a client using apiKeyFrom (e.g. env:ANTHROPIC_API_KEY), and optionally workspaceIdFrom (e.g. env:ANTHROPIC_WORKSPACE_ID) for the anthropic-workspace-id header required by identity-linked keys.

func ResolveAPIKeyFrom

func ResolveAPIKeyFrom(spec string) (string, error)

ResolveAPIKeyFrom parses apiKeyFrom values from project YAML (§7.1), e.g. "env:OPENAI_API_KEY".

func ResolveProviderBaseURL added in v0.4.7

func ResolveProviderBaseURL(cfg spec.ModelProviderConfig, fallback string) string

ResolveProviderBaseURL returns cfg.BaseURL with trailing slashes stripped, or fallback when the alias does not override the endpoint (issue #546).

Types

type ChatMessage

type ChatMessage struct {
	Role        string       `json:"role"`
	Content     string       `json:"content,omitempty"`
	ToolCalls   []ToolCall   `json:"tool_calls,omitempty"`
	ToolResults []ToolResult `json:"tool_results,omitempty"`
}

ChatMessage is one turn in the prompt payload.

type GenerateMeta

type GenerateMeta struct {
	DurationMs       int64   `json:"duration_ms"`
	PromptTokens     int     `json:"prompt_tokens,omitempty"`
	CompletionTokens int     `json:"completion_tokens,omitempty"`
	CostUSD          float64 `json:"cost_usd,omitempty"`
}

GenerateMeta holds duration, token usage, and cost accounting (§13.2 style). For OpenAI, Anthropic, and the mock client, CostUSD is a rough estimate from usage × published per-million token rates when the model is recognized (see tokenUSDPerMillion in cost.go). The mock leaves an explicit non-zero CostUSD unchanged. Unknown model ids stay at 0.

type GenerateRequest

type GenerateRequest struct {
	Model      string        `json:"model"`
	Messages   []ChatMessage `json:"messages"`
	Tools      []ToolDef     `json:"tools,omitempty"`
	ToolChoice string        `json:"tool_choice,omitempty"`
	// MaxTokens caps the completion's output tokens. Zero leaves it to the adapter's own default
	// (Anthropic requires max_tokens on every request; OpenAI omits it, taking the API default). The
	// engine sets it from the agent's constraints.maxTokens (issue #514).
	MaxTokens int `json:"max_tokens,omitempty"`
	// Temperature is the sampling temperature to send to the provider. Nil leaves it unset so the
	// provider default applies; a non-nil value (including 0 for deterministic output) is sent
	// verbatim. Adapters translate it to the provider's request field (issue #388).
	Temperature *float64 `json:"temperature,omitempty"`
	// ResponseFormat, when non-nil, asks the provider to constrain the completion to a JSON Schema
	// ("structured outputs"). Nil leaves the output unconstrained. Adapters that support it translate
	// it to the provider request field (Anthropic output_config.format, OpenAI response_format); the
	// engine sets it for an agent whose constraints require structured output (issue #510).
	ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
}

GenerateRequest is a generation call with optional tool definitions.

func (GenerateRequest) ToolChoiceOrDefault

func (r GenerateRequest) ToolChoiceOrDefault() string

ToolChoiceOrDefault returns ToolChoiceAuto when ToolChoice is unset.

type GenerateResponse

type GenerateResponse struct {
	// Content is assistant message text when the model replies directly.
	Content    string       `json:"content,omitempty"`
	ToolCalls  []ToolCall   `json:"tool_calls,omitempty"`
	StopReason string       `json:"stop_reason,omitempty"`
	Meta       GenerateMeta `json:"meta"`
}

GenerateResponse carries model output, optional tool calls, and accounting metadata.

type MockClient

type MockClient struct {
	Content string
	Meta    *GenerateMeta
	Script  []MockTurn
	// contains filtered or unexported fields
}

MockClient returns deterministic output for tests and offline agent steps (design doc §12.2 F MVP).

When Script is empty, every Generate returns Content with Meta (legacy single-shot behavior) and StopReasonEndTurn, unless req.Tools is non-empty (issue #167 / #168):

  • If a restart-like tool is advertised, the first Generate returns StopReasonToolUse for that tool and the next returns incident status JSON.
  • Otherwise the first Generate returns StopReasonToolUse for the first non-restart tool and the next returns MockClient.Content (or a small JSON object).

When Script is set, each Generate consumes the next turn. After the script is exhausted, Generate returns an error so extra loop iterations fail in tests.

Each call records the request (including Tools) so tests can assert on what the loop sent. MockClient.Reset clears the cursor, recorded requests, and empty-Script hooks when a test reuses one client.

func (*MockClient) CallCount

func (m *MockClient) CallCount() int

CallCount returns how many times Generate has been invoked.

func (*MockClient) Generate

Generate returns the next scripted turn, or the fixed Content when Script is empty.

func (*MockClient) LastRequest

func (m *MockClient) LastRequest() GenerateRequest

LastRequest returns the most recent GenerateRequest, or a zero value if Generate has not been called.

func (*MockClient) Requests

func (m *MockClient) Requests() []GenerateRequest

Requests returns a copy of every GenerateRequest received, in call order.

func (*MockClient) Reset

func (m *MockClient) Reset()

Reset clears recorded requests, the script cursor, and empty-Script tool-loop hooks so one client can be reused across cases. Content, Meta, and Script are left unchanged.

type MockTurn

type MockTurn struct {
	Content    string
	ToolCalls  []ToolCall
	StopReason string
	// Meta is per-call accounting (token counts, cost). When nil, [MockClient.Meta] or a small default is used.
	// Token counts with CostUSD 0 are priced via the B1 table for req.Model (issue #164).
	Meta *GenerateMeta
	// Err, when set, is returned from Generate instead of a response.
	Err error
}

MockTurn is one scripted Generate response for a tool-calling loop (issue #159). StopReason defaults to StopReasonToolUse when ToolCalls is non-empty, otherwise StopReasonEndTurn.

type ModelClient

type ModelClient interface {
	Generate(ctx context.Context, req GenerateRequest) (GenerateResponse, error)
}

ModelClient invokes a chat-capable model (design doc §12.2 F).

type OpenAIClient

type OpenAIClient struct {
	APIKey     string
	BaseURL    string
	HTTPClient *http.Client
	// CostProvider selects the pricing table used to estimate CostUSD.
	// Empty means the OpenAI table (costProviderOpenAI).
	CostProvider string
}

OpenAIClient is an OpenAI-compatible Chat Completions client (design doc §12.2 F). It maps the provider-neutral tool contract to `tools` / `tool_calls` / `role: "tool"` (issue #157).

The same client backs other providers that expose an OpenAI-compatible `/chat/completions` surface — xAI Grok, Google Gemini, and Moonshot Kimi — by pointing BaseURL at their endpoint and setting CostProvider so cost estimation uses that provider's price table (see NewGrokClientFromConfig, NewGeminiClientFromConfig, NewKimiClientFromConfig).

func NewGeminiClientFromConfig

func NewGeminiClientFromConfig(cfg spec.ModelProviderConfig) (*OpenAIClient, error)

NewGeminiClientFromConfig builds a Google Gemini client using apiKeyFrom (e.g. env:GEMINI_API_KEY). Recommended models: gemini-3.1-pro-preview, gemini-3.5-flash, gemini-2.5-flash-lite.

func NewGrokClientFromConfig

func NewGrokClientFromConfig(cfg spec.ModelProviderConfig) (*OpenAIClient, error)

NewGrokClientFromConfig builds an xAI Grok client using apiKeyFrom (e.g. env:XAI_API_KEY). Recommended models: grok-4.6 (flagship), grok-4-fast.

func NewKimiClientFromConfig

func NewKimiClientFromConfig(cfg spec.ModelProviderConfig) (*OpenAIClient, error)

NewKimiClientFromConfig builds a Moonshot Kimi client using apiKeyFrom (e.g. env:MOONSHOT_API_KEY). Recommended models: kimi-k3 (flagship), kimi-k2.7-code.

func NewOpenAIClientFromConfig

func NewOpenAIClientFromConfig(cfg spec.ModelProviderConfig) (*OpenAIClient, error)

NewOpenAIClientFromConfig builds a client using apiKeyFrom (e.g. env:OPENAI_API_KEY) from project YAML.

func (*OpenAIClient) Generate

Generate calls POST /v1/chat/completions on the configured base URL.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry resolves model references using Project.spec.providers.models (design doc §7.1, issue #17), falling back to Terfyn's built-in provider namespaces (issue #430) so a .agent program can select a provider (model anthropic/…) with no providers.models declaration.

func NewRegistry

func NewRegistry(g *spec.ProjectGraph) *Registry

NewRegistry returns a registry from the merged project graph.

func (*Registry) ClientFor

func (r *Registry) ClientFor(modelRef string) (client ModelClient, modelID string, err error)

ClientFor resolves modelRef in the form "namespace/model_id" (e.g. "openai/gpt-4.1"). The returned modelID is the segment after the first slash and should be passed as GenerateRequest.Model.

type ResponseFormat added in v0.4.2

type ResponseFormat struct {
	// Name is a short schema identifier some providers require (OpenAI json_schema.name); adapters
	// that do not need it (Anthropic) ignore it. It must match ^[a-zA-Z0-9_-]{1,64}$.
	Name string `json:"name,omitempty"`
	// Schema is the JSON Schema the completion must conform to.
	Schema json.RawMessage `json:"schema,omitempty"`
}

ResponseFormat requests provider-enforced structured output: the completion must be a JSON value conforming to Schema. Schema must be a JSON Schema object and, for providers that require it (OpenAI, and Anthropic's strict subset), an object schema with additionalProperties:false and its properties listed in required — otherwise the provider rejects the request (issue #510).

type ToolCall

type ToolCall struct {
	ID        string          `json:"id"`
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
}

ToolCall is a model-issued request to invoke a tool before the turn ends.

type ToolDef

type ToolDef struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty"`
}

ToolDef describes one callable tool exposed to the model (provider-neutral JSON Schema parameters).

type ToolResult

type ToolResult struct {
	ToolCallID string `json:"tool_call_id"`
	Content    string `json:"content"`
	// IsError marks this result as a failed tool call. Providers that model it (Anthropic's
	// tool_result.is_error) receive the flag so the model treats the content as an error observation
	// to correct from, not a successful output; providers without the concept (OpenAI) ignore it and
	// carry the content alone. A failed or malformed tool call must always be answered by a
	// well-formed tool_result — never left as a dangling tool_use that poisons the next request (#524).
	IsError bool `json:"is_error,omitempty"`
}

ToolResult carries one tool execution result back to the model on a follow-up turn. Attach results on ChatMessage via ChatMessage.ToolResults; keep ChatMessage.Role and ChatMessage.Content for ordinary text turns. Replay the originating assistant turn with ChatMessage.ToolCalls so providers can emit native tool-call blocks before tool results (issue #156, #157).

Directories

Path Synopsis
Package anthropic implements the Anthropic Messages API client (design doc §7.1, issue #69).
Package anthropic implements the Anthropic Messages API client (design doc §7.1, issue #69).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL