provider

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 16, 2026 License: MIT Imports: 45 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// DefaultBedrockModel is the default cross-region inference profile for Bedrock.
	DefaultBedrockModel = "anthropic.claude-sonnet-5"
	// DefaultVertexModel is the default model for Vertex AI.
	DefaultVertexModel = "claude-sonnet-5"
)

Default model IDs used across constructor and env-config functions.

View Source
const (
	MinOverrideContextWindow = 8192
	MaxOverrideContextWindow = 10000000
)

Variables

This section is empty.

Functions

func AsAPIError

func AsAPIError(err error) (*types.APIError, bool)

AsAPIError extracts a *types.APIError from an error value, if present.

func CanonicalProviderName

func CanonicalProviderName(name string) string

CanonicalProviderName normalizes a provider identifier.

func CatalogDefaultModel

func CatalogDefaultModel(provider, fallback string) string

CatalogDefaultModel returns the catalog default model for a provider, falling back to the supplied value when the provider has no registered default.

func CompatibleProviderDisplayName

func CompatibleProviderDisplayName(displayName, baseURL string) string

CompatibleProviderDisplayName applies the requested hostname fallback for a user-created gateway.

func CostCurrencySymbol

func CostCurrencySymbol(currency string) string

CostCurrencySymbol returns a compact display symbol for a billing currency.

func CreateStreamAttempt

func CreateStreamAttempt(ctx context.Context, controller *AttemptController, p Provider, params Params) (<-chan types.StreamEvent, error)

CreateStreamAttempt charges exactly one response-stream attempt. Any RetryProvider nested inside the call owns a separate HTTP request budget.

func CredentialLookupNames

func CredentialLookupNames(providerName string) []string

CredentialLookupNames returns the canonical credential-store key.

func DefaultMaxOutputTokens

func DefaultMaxOutputTokens(providerName, model string) int

DefaultMaxOutputTokens returns the request-side output budget used when the caller did not provide an explicit limit. Model MaxOutput metadata remains a capability ceiling and must not be used as the request default.

func DefaultReasoningEffort

func DefaultReasoningEffort(efforts []string) string

DefaultReasoningEffort returns the catalog default for a model's selectable reasoning tiers. Medium is preferred when available; provider-defined lists that do not include it retain their first advertised tier as the default.

func DefaultReasoningEffortForModel

func DefaultReasoningEffortForModel(model ModelInfo) string

DefaultReasoningEffortForModel returns the model-specific default when it is one of the advertised tiers, then falls back to the shared selection rule.

func DefaultRequestMaxOutput

func DefaultRequestMaxOutput(providerName, model string) int

DefaultRequestMaxOutput returns a model/provider-specific request default. Zero means the caller should retain its shared default or provider default. Explicit application or provider configuration must take precedence over this value.

func FormatContextWindow

func FormatContextWindow(tokens int) string

FormatContextWindow formats a token window using common rounded labels. It intentionally maps binary/token-exact limits like 1,048,576 and 262,144 to user-facing labels such as 1M and 256K, matching provider documentation.

func FormatDebugEvent

func FormatDebugEvent(event DebugEvent) (string, error)

FormatDebugEvent renders a debug exchange half as readable JSON.

func Is529Error

func Is529Error(err error) bool

Is529Error reports whether err is an HTTP 529 overloaded error.

func IsAttemptLimit

func IsAttemptLimit(err error) bool

IsAttemptLimit reports whether a generation has consumed its complete raw provider-call budget.

func IsPromptTooLong

func IsPromptTooLong(err error) bool

IsPromptTooLong reports whether err signals that the prompt is too long to be processed (distinct from retryable context-overflow).

func IsRetryable

func IsRetryable(err error) bool

IsRetryable reports whether the typed contract permits a retry. The caller must still ask its generation-scoped AttemptController for budget and delay.

func LookupMaxContext

func LookupMaxContext(model string) int

LookupMaxContext returns the context window size for the given model name. It resolves through the default ModelCatalog so context metadata comes from the same source as /model and the model picker.

func LookupMaxOutput

func LookupMaxOutput(model string) int

LookupMaxOutput returns the maximum output token count for the given model. Zero means the catalog does not know a provider/model-specific maximum.

func NormalizeModelIdentifier

func NormalizeModelIdentifier(id string) string

NormalizeModelIdentifier implements the gateway matching rule: comparisons are case-insensitive and ignore underscore and hyphen separators.

func OpenDebugFile

func OpenDebugFile(path string) (*os.File, error)

OpenDebugFile opens path for append and restricts it to the current user. Debug output can contain system prompts, local file contents, and tool data.

func OverrideKey

func OverrideKey(providerName, modelID string) string

func ResolveRequestMaxOutput

func ResolveRequestMaxOutput(providerName, model string, configured int) int

ResolveRequestMaxOutput applies a provider/model default only when the caller did not explicitly configure a positive output budget.

func SetRuntimeModelOverrides

func SetRuntimeModelOverrides(overrides ModelOverrides)

func TryFallbackTransport

func TryFallbackTransport(p Provider) (from, to string, activated bool)

TryFallbackTransport resolves provider decorators and activates the next safe streaming transport, if one is available. Transport names are stable protocol identifiers intended for diagnostics and localized copy arguments.

func ValidOverrideContextWindow

func ValidOverrideContextWindow(value int) bool

func ValidateParams

func ValidateParams(p Provider, params Params) error

ValidateParams checks that the provider supports the capabilities required by params. Returns an error if a critical capability mismatch is detected, preventing silent degradation.

func WithAttemptController

func WithAttemptController(ctx context.Context, controller *AttemptController) context.Context

WithAttemptController binds a generation budget to all nested provider decorators participating in the request.

func WithDebugCall

func WithDebugCall(ctx context.Context, kind DebugCallKind, metadata map[string]any) context.Context

WithDebugCall marks why an LLM request is being made. Nested calls preserve existing metadata and let the most specific non-empty values win.

func WithLocalProviderRequestID

func WithLocalProviderRequestID(ctx context.Context, requestID string) context.Context

WithLocalProviderRequestID correlates the runtime attempt with provider and presentation diagnostics. The identifier is generated locally and carries no provider-controlled content.

func WithRetryObserver

func WithRetryObserver(ctx context.Context, observer func(RetryEvent)) context.Context

WithRetryObserver attaches a request-scoped retry observer. It is used by interactive clients to project retry progress without changing provider configuration shared by other sessions.

Types

type APIStyle

type APIStyle string

APIStyle identifies the compatibility protocol exposed by an aggregate provider or gateway.

const (
	APIStyleOpenAI    APIStyle = "openai"
	APIStyleAnthropic APIStyle = "anthropic"
)

func ParseAPIStyle

func ParseAPIStyle(value string) APIStyle

ParseAPIStyle normalizes persisted and user-supplied protocol names. Unknown values intentionally fall back to OpenAI, the default in the setup flow.

func (APIStyle) DisplayName

func (style APIStyle) DisplayName() string

DisplayName returns the protocol's stable product name. Protocol and brand names intentionally are not translated.

type AnthropicProvider

type AnthropicProvider struct {
	// contains filtered or unexported fields
}

AnthropicProvider wraps the official Anthropic SDK as a Provider

func NewAnthropic

func NewAnthropic(cfg Config) *AnthropicProvider

NewAnthropic creates a Provider backed by the Anthropic API

func (*AnthropicProvider) Capabilities

func (p *AnthropicProvider) Capabilities() ProviderCapabilities

Capabilities implements CapabilityProvider for AnthropicProvider.

func (*AnthropicProvider) CountTokens

func (p *AnthropicProvider) CountTokens(ctx context.Context, content string) (int, error)

CountTokens uses Anthropic's token-count endpoint for near-limit Read validation. The content is sent as one user text block, matching the TS countTokensWithAPI path for file content.

func (*AnthropicProvider) CreateStream

func (p *AnthropicProvider) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

func (*AnthropicProvider) ModelID

func (p *AnthropicProvider) ModelID() string

func (*AnthropicProvider) Name

func (p *AnthropicProvider) Name() string

type AttemptController

type AttemptController struct {
	// contains filtered or unexported fields
}

AttemptController owns one retry budget. The runtime creates a generation- scoped controller for stream reconnects, while RetryProvider creates a fresh request-scoped controller for each attempt to establish a stream.

func AttemptControllerForProvider

func AttemptControllerForProvider(p Provider, fallback ...RetryConfig) *AttemptController

AttemptControllerForProvider creates a controller from the retry policy of the provider being sampled. ProviderRef wrappers are resolved recursively. Raw providers use fallback when supplied, otherwise the production default.

func NewAttemptController

func NewAttemptController(config RetryConfig) *AttemptController

NewAttemptController creates a generation-scoped controller. A zero-value configuration receives the same production defaults as RetryProvider.

func (*AttemptController) Attempts

func (c *AttemptController) Attempts() int

Attempts reports how many raw provider calls have begun.

func (*AttemptController) CanRetry

func (c *AttemptController) CanRetry(err error) bool

CanRetry combines the typed provider disposition with this controller's scope-local budget.

func (*AttemptController) MaxAttempts

func (c *AttemptController) MaxAttempts() int

MaxAttempts is the total number of attempts in this controller's scope.

func (*AttemptController) RetryDelay

func (c *AttemptController) RetryDelay(err error) (time.Duration, bool)

RetryDelay returns the delay before another attempt. It applies exponential backoff with ±10% jitter. Retry-After is authoritative when longer than the computed delay, as it is in Codex CLI. false means the failure is permanent or this scope's budget is exhausted.

func (*AttemptController) Wait

func (c *AttemptController) Wait(ctx context.Context, delay time.Duration) error

Wait blocks for a retry delay while preserving context cancellation.

type AttemptErrorContract

type AttemptErrorContract struct {
	Stage        types.ProviderErrorStage
	Class        types.ProviderErrorClass
	ReplaySafety types.ProviderReplaySafety
}

AttemptErrorContract is the complete machine-actionable retry disposition for one failed raw provider attempt. It is intentionally independent from provider-controlled diagnostic prose.

func ClassifyAttemptError

func ClassifyAttemptError(err error) AttemptErrorContract

ClassifyAttemptError derives the typed retry contract using structured fields only. Message is diagnostic data and is never consulted. Explicit metadata supplied by a transaction wrapper has highest priority, followed by API code/type/status and concrete Go transport interfaces.

func (AttemptErrorContract) Retryable

func (c AttemptErrorContract) Retryable() bool

Retryable reports whether policy permits another raw call. Attempt budget, delay, and authentication refresh are still owned by AttemptController.

type AttemptLimitError

type AttemptLimitError struct {
	MaxAttempts int
	Attempts    int
	Cause       error
}

AttemptLimitError retains the final transport cause while making exhaustion machine-detectable to callers, preventing a second retry layer from starting another loop.

func (*AttemptLimitError) AttemptErrorClass

func (e *AttemptLimitError) AttemptErrorClass() types.ProviderErrorClass

func (*AttemptLimitError) AttemptErrorStage

func (e *AttemptLimitError) AttemptErrorStage() types.ProviderErrorStage

AttemptErrorStage/Class/ReplaySafety make a request-budget exhaustion a terminal boundary for the outer stream loop. The preserved cause remains available through errors.Is/As, but it must not start a second request retry loop and multiply the configured budget.

func (*AttemptLimitError) AttemptReplaySafety

func (e *AttemptLimitError) AttemptReplaySafety() types.ProviderReplaySafety

func (*AttemptLimitError) Error

func (e *AttemptLimitError) Error() string

func (*AttemptLimitError) Unwrap

func (e *AttemptLimitError) Unwrap() error

type BedrockConfig

type BedrockConfig struct {
	// Region is the AWS region (e.g. "us-east-1"). Defaults to AWS_REGION / AWS_DEFAULT_REGION env.
	Region string

	// Model is the Bedrock model ID (e.g. "anthropic.claude-3-5-sonnet-20241022-v2:0").
	// Supports foundation models, cross-region inference profiles (e.g. "us.anthropic.claude-..."),
	// and ARN-format inference profiles.
	Model string

	// MaxTokens overrides the default max output tokens.
	MaxTokens int

	// Timeout in seconds for requests.
	Timeout int

	// StaticCredentials, if set, skips the default credential chain.
	// Populated from AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY + AWS_SESSION_TOKEN env vars.
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string

	// BearerToken is set from AWS_BEARER_TOKEN_BEDROCK for API-key-style auth.
	BearerToken string

	// BaseURL overrides the Bedrock runtime endpoint (ANTHROPIC_BEDROCK_BASE_URL).
	// Must be empty, start with "https://", or start with "http://localhost" (for testing).
	BaseURL string
}

BedrockConfig holds AWS-specific configuration for the Bedrock provider.

func BedrockConfigFromEnv

func BedrockConfigFromEnv() BedrockConfig

BedrockConfigFromEnv reads Bedrock configuration from environment variables.

Variables read:

  • AWS_REGION / AWS_DEFAULT_REGION — AWS region (falls back to "us-east-1")
  • AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN — static credentials
  • AWS_BEARER_TOKEN_BEDROCK — bearer token for API-key-style auth
  • ANTHROPIC_BEDROCK_BASE_URL — custom endpoint override
  • BEDROCK_MODEL — model ID override

type BedrockProvider

type BedrockProvider struct {
	// contains filtered or unexported fields
}

BedrockProvider wraps the Anthropic SDK with AWS Bedrock transport.

func NewBedrock

func NewBedrock(ctx context.Context, cfg BedrockConfig) (*BedrockProvider, error)

NewBedrock creates a Provider backed by Amazon Bedrock.

Authentication priority (mirrors the Anthropic SDK behaviour):

  1. AWS_BEARER_TOKEN_BEDROCK env / cfg.BearerToken — bearer token auth
  2. cfg.AccessKeyID + cfg.SecretAccessKey — static SigV4 credentials
  3. Default AWS credential chain (env → shared config → EC2 IMDS → …)

ctx is passed to the AWS credential chain (e.g. for IMDS metadata fetches).

func (*BedrockProvider) Capabilities

func (p *BedrockProvider) Capabilities() ProviderCapabilities

Capabilities implements CapabilityProvider.

func (*BedrockProvider) CreateStream

func (p *BedrockProvider) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

CreateStream uses the shared Anthropic stream implementation with our Bedrock client.

func (*BedrockProvider) ModelID

func (p *BedrockProvider) ModelID() string

func (*BedrockProvider) Name

func (p *BedrockProvider) Name() string

type CacheRoutingMode

type CacheRoutingMode string

CacheRoutingMode describes how a cache lineage is lowered onto the wire. It is deliberately separate from CacheControl, which means Anthropic-style content-block cache breakpoints rather than request routing affinity.

const (
	CacheRoutingNone                     CacheRoutingMode = ""
	CacheRoutingPromptCacheKey           CacheRoutingMode = "prompt_cache_key"
	CacheRoutingPromptCacheKeyBestEffort CacheRoutingMode = "prompt_cache_key_best_effort"
	CacheRoutingDeepSeekUserID           CacheRoutingMode = "deepseek_user_id"
)

type CacheRoutingPreference

type CacheRoutingPreference string

CacheRoutingPreference is an operator override for request-level cache routing fields. Auto uses the provider's documented or best-effort policy, On upgrades best-effort prompt_cache_key to strict delivery, and Off omits routing fields without changing native content-block cache_control support.

const (
	CacheRoutingAuto CacheRoutingPreference = ""
	CacheRoutingOn   CacheRoutingPreference = "on"
	CacheRoutingOff  CacheRoutingPreference = "off"
)

func ParseCacheRoutingPreference

func ParseCacheRoutingPreference(value string) CacheRoutingPreference

ParseCacheRoutingPreference accepts the persisted auto/on/off spelling. Unknown values safely retain the default automatic policy.

type CapabilityProvider

type CapabilityProvider interface {
	Capabilities() ProviderCapabilities
}

CapabilityProvider is an optional interface for providers that can report their capabilities. Callers should type-assert: if cp, ok := p.(CapabilityProvider); ok { ... }

type CapabilitySupport

type CapabilitySupport uint8

CapabilitySupport distinguishes an explicitly supported contract from an unknown endpoint. Experimental wire formats must require Supported; Unknown is deliberately fail-closed.

const (
	CapabilityUnknown CapabilitySupport = iota
	CapabilityUnsupported
	CapabilitySupported
)

type CloseProvider

type CloseProvider interface {
	Close() error
}

CloseProvider is an optional lifecycle implemented by providers that own persistent transports. Callers must invoke it only after in-flight streams have drained.

type CompatibleModelRequest

type CompatibleModelRequest struct {
	Provider string
	APIStyle APIStyle
	BaseURL  string
	APIKey   string
}

CompatibleModelRequest contains the endpoint inputs needed for authenticated model discovery.

type CompatibleProviderDefinition

type CompatibleProviderDefinition struct {
	Name        string
	DisplayName string
	Popularity  int
	BaseURLs    map[APIStyle]string
	UserDefined bool
}

CompatibleProviderDefinition describes a multi-model endpoint implemented through one of the shared compatibility protocols.

type Config

type Config struct {
	ProviderName string
	APIStyle     APIStyle

	// APIFormat is an explicit OpenAI-family wire override. Empty selects the
	// native provider's cataloged format; compatible providers may negotiate a
	// catalog-inferred format. Supported values are responses and chat-completions.
	APIFormat string
	// ResponsesSemantics declares the upstream wire contract independently of
	// the transport URL. This is required when the official OpenAI API is
	// reached through a content-blind benchmark proxy: changing only URL/auth
	// must not silently downgrade the request to generic-compatible semantics.
	ResponsesSemantics ResponsesSemantics
	// ResponsesWebSocket is an explicit endpoint capability assertion. Unknown
	// and Unsupported keep the documented HTTP/SSE transport. Supported enables
	// the production WebSocket transport only for the public Responses profile;
	// it is never inferred from a hostname or model name.
	ResponsesWebSocket        CapabilitySupport
	APIKey                    string
	AuthToken                 string
	BaseURL                   string
	Model                     string
	MaxTokens                 int
	Headers                   map[string]string
	Timeout                   int // seconds
	DisableStrictTools        bool
	DisablePromptCacheOptions bool
	CacheRoutingPreference    CacheRoutingPreference
}

Config holds provider-agnostic configuration

func ResolveCredentialConfig

func ResolveCredentialConfig(r *ProviderRegistry, providerName string) (Config, error)

ResolveCredentialConfig builds provider config from the attached credential store. OpenAI OAuth entries use the ChatGPT Codex backend, matching Codex's ChatGPT-login path; API-key entries use the public OpenAI API.

type ConnectionDetail

type ConnectionDetail struct {
	Provider        string
	DisplayName     string
	State           ConnectionState
	Kind            ConnectionKind
	Source          string
	DetailKey       i18n.Key
	DetailArgs      []any
	CanSelectModels bool
	CanConnect      bool
	SetupHintKey    i18n.Key
	SetupHintArgs   []any
}

ConnectionDetail is the canonical provider connection status consumed by commands and TUI code. CanSelectModels is intentionally separate from State: local providers may be selectable without being verified as connected.

func (ConnectionDetail) DetailText

func (d ConnectionDetail) DetailText(lang i18n.Language) string

func (ConnectionDetail) SetupHintText

func (d ConnectionDetail) SetupHintText(lang i18n.Language) string

type ConnectionKind

type ConnectionKind string

ConnectionKind identifies the authentication or backend class.

const (
	ConnectionKindAPIKey         ConnectionKind = "api_key"
	ConnectionKindOAuth          ConnectionKind = "oauth"
	ConnectionKindEnv            ConnectionKind = "env"
	ConnectionKindAWSCredentials ConnectionKind = "aws_credentials"
	ConnectionKindGCPADC         ConnectionKind = "gcp_adc"
	ConnectionKindLocalService   ConnectionKind = "local_service"
	ConnectionKindNone           ConnectionKind = "none"
)

type ConnectionState

type ConnectionState string

ConnectionState describes the user's effective ability to use a provider.

const (
	ConnectionStateConnected     ConnectionState = "connected"
	ConnectionStateNotConfigured ConnectionState = "not_configured"
	ConnectionStateLocal         ConnectionState = "local_unverified"
	ConnectionStateUnknown       ConnectionState = "unknown"
)

type CredentialEntry

type CredentialEntry struct {
	// Provider is the canonical provider name (e.g. "anthropic", "openai").
	Provider string `json:"provider"`

	// AuthMethod describes how the credential was obtained.
	// Values: "api_key", "oauth"
	AuthMethod string `json:"auth_method"`

	// APIKey is the provider API key, or an optional exchanged API key for OAuth entries.
	APIKey string `json:"api_key,omitempty"`

	// AccessToken is the OAuth access token (for "oauth" auth method).
	AccessToken string `json:"access_token,omitempty"`

	// RefreshToken is the OAuth refresh token (for "oauth" auth method).
	RefreshToken string `json:"refresh_token,omitempty"`

	// ExpiresAt is the expiry time for OAuth tokens.
	ExpiresAt time.Time `json:"expires_at,omitempty"`

	// AccountID is the ChatGPT workspace/account identifier used by the Codex backend.
	AccountID string `json:"account_id,omitempty"`

	// ChatGPTPlanType is informational metadata parsed from the OpenAI ID token.
	ChatGPTPlanType string `json:"chatgpt_plan_type,omitempty"`

	// ChatGPTAccountIsFedRAMP routes workspace traffic through the FedRAMP edge.
	ChatGPTAccountIsFedRAMP bool `json:"chatgpt_account_is_fedramp,omitempty"`

	// BaseURL is an optional custom API base URL.
	BaseURL string `json:"base_url,omitempty"`

	// APIStyle selects the compatibility protocol used for model discovery and
	// inference. Empty entries predate compatible aggregate providers.
	APIStyle APIStyle `json:"api_style,omitempty"`
	// APIFormat is an explicit OpenAI-family wire override. It is deliberately
	// separate from Models[].APIFormat, which is discovered catalog metadata.
	APIFormat string `json:"api_format,omitempty"`
	// DisableStrictTools is an explicit compatibility escape hatch for an
	// OpenAI Responses endpoint that is reached through a gateway which rejects
	// strict function schemas. The endpoint identity remains OpenAI; only schema
	// strictness is relaxed.
	DisableStrictTools bool `json:"disable_strict_tools,omitempty"`
	// DisablePromptCacheOptions preserves prompt_cache_key routing while
	// omitting public-API cache option and breakpoint extensions rejected by
	// some content-blind gateways.
	DisablePromptCacheOptions bool `json:"disable_prompt_cache_options,omitempty"`

	// DisplayName and UserDefined persist providers created through the generic
	// gateway flow without requiring a second configuration file.
	DisplayName string `json:"display_name,omitempty"`
	UserDefined bool   `json:"user_defined,omitempty"`

	// Models caches the most recently discovered catalog so a configured
	// provider remains selectable before its next online refresh.
	Models []ModelInfo `json:"models,omitempty"`

	// LastUsed records when this credential was last used.
	LastUsed time.Time `json:"last_used"`
}

CredentialEntry stores authentication credentials for a single provider.

func RefreshOpenAIOAuthCredential

func RefreshOpenAIOAuthCredential(ctx context.Context, entry CredentialEntry) (CredentialEntry, error)

RefreshOpenAIOAuthCredential refreshes an OpenAI OAuth credential and exchanges the returned id_token for a usable OpenAI API key.

type CredentialStore

type CredentialStore struct {
	// contains filtered or unexported fields
}

CredentialStore manages provider credentials with file-based persistence. Credentials are stored in ~/.luban-code/auth.json with 0600 permissions.

Thread-safe for concurrent access.

func DefaultCredentialStore

func DefaultCredentialStore() (*CredentialStore, error)

DefaultCredentialStore returns the singleton CredentialStore.

func NewCredentialStore

func NewCredentialStore() (*CredentialStore, error)

NewCredentialStore creates a CredentialStore backed by ~/.luban-code/auth.json. It loads existing credentials from disk if the file exists.

func NewCredentialStoreAt

func NewCredentialStoreAt(path string) (*CredentialStore, error)

NewCredentialStoreAt creates a CredentialStore at a specific file path. Used for testing and custom configurations.

func (*CredentialStore) All

func (s *CredentialStore) All() []CredentialEntry

All returns all credential entries, sorted by provider name.

func (*CredentialStore) Delete

func (s *CredentialStore) Delete(provider string) error

Delete removes a credential entry and persists the change.

func (*CredentialStore) Get

func (s *CredentialStore) Get(provider string) (CredentialEntry, bool)

Get returns the credential entry for a provider.

func (*CredentialStore) HasCredentials

func (s *CredentialStore) HasCredentials(provider string) bool

HasCredentials checks if valid credentials exist for a provider. For API keys, checks non-empty. For OAuth, checks non-expired access token.

func (*CredentialStore) Set

func (s *CredentialStore) Set(entry CredentialEntry) error

Set saves or updates a credential entry and persists to disk.

func (*CredentialStore) UpdateLastUsed

func (s *CredentialStore) UpdateLastUsed(provider string)

UpdateLastUsed updates the LastUsed timestamp for a provider.

type DebugCallKind

type DebugCallKind string

DebugCallKind identifies the runtime operation that initiated an LLM call.

const (
	DebugCallConversation DebugCallKind = "conversation"
	DebugCallCompaction   DebugCallKind = "compaction"
)

type DebugEvent

type DebugEvent struct {
	ID       uint64         `json:"id"`
	Phase    DebugPhase     `json:"phase"`
	Kind     DebugCallKind  `json:"kind"`
	Provider string         `json:"provider"`
	Model    string         `json:"model"`
	Metadata map[string]any `json:"metadata,omitempty"`
	Request  *DebugRequest  `json:"request,omitempty"`
	Response *DebugResponse `json:"response,omitempty"`
}

DebugEvent is one half of a request/response pair. ID correlates both halves.

type DebugObserver

type DebugObserver func(DebugEvent)

DebugObserver receives opt-in, developer-facing snapshots of LLM calls. HTTP headers and credentials are intentionally outside this abstraction.

func NewDebugWriterObserver

func NewDebugWriterObserver(w io.Writer) DebugObserver

NewDebugWriterObserver writes complete formatted events serially to w.

type DebugPhase

type DebugPhase string

DebugPhase identifies which side of an LLM exchange a debug event describes.

const (
	DebugPhaseRequest  DebugPhase = "request"
	DebugPhaseResponse DebugPhase = "response"
)

type DebugRequest

type DebugRequest struct {
	Stream                   bool                         `json:"stream"`
	Model                    string                       `json:"model"`
	MaxTokens                int                          `json:"max_tokens"`
	MaxOutputTokensOverride  int                          `json:"max_output_tokens_override,omitempty"`
	EffectiveMaxOutputTokens int                          `json:"effective_max_output_tokens,omitempty"`
	CatalogMaxOutputTokens   int                          `json:"catalog_max_output_tokens,omitempty"`
	System                   string                       `json:"system,omitempty"`
	SystemBlocks             []prompt.SystemPromptBlock   `json:"system_blocks,omitempty"`
	Messages                 []types.Message              `json:"messages"`
	Tools                    []types.ToolDefinition       `json:"tools,omitempty"`
	ExtraToolSchemas         []types.ServerToolDefinition `json:"extra_tool_schemas,omitempty"`
	ToolChoice               *ToolChoice                  `json:"tool_choice,omitempty"`
	Thinking                 *ThinkingConfig              `json:"thinking,omitempty"`
	TaskBudget               *TaskBudget                  `json:"task_budget,omitempty"`
	Conversation             string                       `json:"conversation,omitempty"`
	PreviousResponseID       string                       `json:"previous_response_id,omitempty"`
	Truncation               string                       `json:"truncation,omitempty"`
	PromptCacheKey           string                       `json:"prompt_cache_key,omitempty"`
	ReasoningEffort          string                       `json:"reasoning_effort,omitempty"`
	ServiceTier              ServiceTier                  `json:"service_tier,omitempty"`
	UsePromptCache           bool                         `json:"use_prompt_cache,omitempty"`
}

DebugRequest is the complete provider-neutral payload handed to the active provider after prompt construction, history preparation, and compaction.

type DebugResponse

type DebugResponse struct {
	Message           *types.Message                   `json:"message,omitempty"`
	Usage             *types.Usage                     `json:"usage,omitempty"`
	StopReason        *types.StopReason                `json:"stop_reason,omitempty"`
	ResponseID        string                           `json:"response_id,omitempty"`
	SystemFingerprint string                           `json:"system_fingerprint,omitempty"`
	Error             string                           `json:"error,omitempty"`
	FailureDiagnostic *types.ProviderFailureDiagnostic `json:"failure_diagnostic,omitempty"`
}

DebugResponse is the semantic model result reconstructed from the provider stream. Token-level deltas are collapsed into content and tool-use blocks.

type DisconnectedProvider

type DisconnectedProvider struct {
	// contains filtered or unexported fields
}

DisconnectedProvider is a placeholder provider that lets the UI boot before credentials are configured. Query attempts fail with a guided message until the user connects a real provider.

func NewDisconnectedProvider

func NewDisconnectedProvider(name, model, message string) *DisconnectedProvider

func (*DisconnectedProvider) CreateStream

func (p *DisconnectedProvider) CreateStream(_ context.Context, _ Params) (<-chan types.StreamEvent, error)

func (*DisconnectedProvider) ModelID

func (p *DisconnectedProvider) ModelID() string

func (*DisconnectedProvider) Name

func (p *DisconnectedProvider) Name() string

type FallbackTriggeredError

type FallbackTriggeredError struct {
	OriginalModel string
	FallbackModel string
	Cause         error
}

FallbackTriggeredError signals that the current model should be abandoned and the same request retried with FallbackModel.

func AsFallbackTriggeredError

func AsFallbackTriggeredError(err error) (*FallbackTriggeredError, bool)

AsFallbackTriggeredError extracts fallback metadata from either the provider typed error or the stream/API error shape used by providers that can only surface errors inside StreamEvent.

func (*FallbackTriggeredError) Error

func (e *FallbackTriggeredError) Error() string

func (*FallbackTriggeredError) Unwrap

func (e *FallbackTriggeredError) Unwrap() error

type ModelCatalog

type ModelCatalog struct {
	// contains filtered or unexported fields
}

ModelCatalog is a registry of known models and their metadata. It is thread-safe for concurrent reads after initialization.

func DefaultCatalog

func DefaultCatalog() *ModelCatalog

DefaultCatalog returns a ModelCatalog pre-populated with generated provider metadata plus local-only model identifiers.

func NewModelCatalog

func NewModelCatalog() *ModelCatalog

NewModelCatalog creates an empty ModelCatalog.

func (*ModelCatalog) All

func (c *ModelCatalog) All() []ModelInfo

All returns every model in the catalog, sorted by provider then ID.

func (*ModelCatalog) ApplyOverrides

func (c *ModelCatalog) ApplyOverrides(overrides ModelOverrides)

func (*ModelCatalog) Count

func (c *ModelCatalog) Count() int

Count returns the total number of models in the catalog.

func (*ModelCatalog) DefaultForProvider

func (c *ModelCatalog) DefaultForProvider(provider string) string

DefaultForProvider returns the default model ID for a given provider.

func (*ModelCatalog) Get

func (c *ModelCatalog) Get(id string) (ModelInfo, bool)

Get returns a model by its exact ID.

func (*ModelCatalog) GetForProvider

func (c *ModelCatalog) GetForProvider(provider, id string) (ModelInfo, bool)

GetForProvider returns a model by provider + exact model ID.

func (*ModelCatalog) ListByProvider

func (c *ModelCatalog) ListByProvider(provider string) []ModelInfo

ListByProvider returns all models for a given provider, sorted by ID.

func (*ModelCatalog) ModelIDsByProvider

func (c *ModelCatalog) ModelIDsByProvider(provider string) []string

ModelIDsByProvider returns canonical model IDs for a provider.

func (*ModelCatalog) Register

func (c *ModelCatalog) Register(m ModelInfo)

Register adds or updates a model in the catalog.

func (*ModelCatalog) RemoveProvider

func (c *ModelCatalog) RemoveProvider(provider string)

RemoveProvider deletes every model associated with provider.

func (*ModelCatalog) ReplaceProviderModels

func (c *ModelCatalog) ReplaceProviderModels(provider string, models []ModelInfo)

ReplaceProviderModels atomically replaces all catalog entries belonging to provider. It is used when a compatible endpoint refreshes its model list.

func (*ModelCatalog) Resolve

func (c *ModelCatalog) Resolve(id string) (ModelInfo, bool)

Resolve looks up a model by exact ID first, then by longest prefix match. This handles versioned model names like "claude-sonnet-4-20250514" matching "claude-sonnet-4".

func (*ModelCatalog) ResolveForProvider

func (c *ModelCatalog) ResolveForProvider(provider, id string) (ModelInfo, bool)

ResolveForProvider looks up a model within a specific provider, trying exact match first and then longest-prefix match.

type ModelInfo

type ModelInfo struct {
	// ID is the canonical model identifier (e.g. "claude-sonnet-4-20250514").
	ID string

	// Aliases lists current provider identifiers that resolve to ID without
	// being shown as duplicate entries in model pickers.
	Aliases []string

	// Name is the human-readable display name (e.g. "Claude Sonnet 4").
	Name string

	// Provider is the provider that owns this model (e.g. "anthropic").
	Provider string

	// ContextWindow is the maximum input context window in tokens.
	ContextWindow int

	// MaxOutput is the maximum output tokens the model can generate.
	// 0 means unknown / provider-default.
	MaxOutput int

	// CostPer1MIn is the cost per 1M input tokens in the model's native billing currency.
	CostPer1MIn float64

	// CostPer1MOut is the cost per 1M output tokens in the model's native billing currency.
	CostPer1MOut float64

	// CacheReadPer1M is the cost per 1M cache-read tokens in the model's native billing currency. 0 = no caching.
	CacheReadPer1M float64

	// CacheCreatePer1M is the cost per 1M cache-creation tokens in the model's native billing currency. 0 = no caching.
	CacheCreatePer1M float64

	// CostCurrency is the ISO 4217 billing currency for all Cost* fields.
	CostCurrency string

	// CanReason indicates the model supports extended thinking / reasoning.
	CanReason bool

	// ReasoningEfforts lists selectable reasoning effort tiers for providers
	// whose API supports a discrete effort parameter. Empty means no secondary
	// effort picker should be shown for this model.
	ReasoningEfforts []string

	// DefaultReasoningEffort is the provider-documented default selectable
	// effort. Empty retains the shared catalog fallback.
	DefaultReasoningEffort string

	// CanUseTools indicates the model supports function/tool calling.
	CanUseTools bool

	// CanSeeImages indicates the model supports vision / image inputs.
	CanSeeImages bool

	// CacheControl indicates the model supports prompt caching.
	CacheControl bool

	// APIFormat is the API protocol: "messages", "chat-completions", "responses".
	APIFormat string

	// IsDefault is true if this is the default model for its provider.
	IsDefault bool

	// ContextOverridden is true when ContextWindow was changed by user settings.
	ContextOverridden bool
}

ModelInfo describes a model's capabilities, pricing, and metadata. It consolidates data previously spread across modelContextWindows and cost/pricing.

func (ModelInfo) BillingCurrency

func (m ModelInfo) BillingCurrency() string

BillingCurrency returns the normalized billing currency for this model.

type ModelOverride

type ModelOverride struct {
	ContextWindow *int `json:"context_window,omitempty"`
	MaxOutput     *int `json:"max_output,omitempty"`
}

type ModelOverrides

type ModelOverrides map[string]ModelOverride

func RuntimeModelOverrides

func RuntimeModelOverrides() ModelOverrides

type OAuthHook

type OAuthHook interface {
	// LoadAccessToken returns a valid access token, refreshing if needed.
	// Returns ("", nil) if no credentials are stored.
	LoadAccessToken(ctx context.Context) (string, error)

	// OnAuthError is called when a 401 is received. Returns true if
	// credentials were successfully refreshed and a retry is warranted.
	OnAuthError() bool
}

OAuthHook is an optional interface for integrating OAuth credential management without creating an import cycle between provider and auth. The hook is set from outside the provider package (e.g., main.go or a bootstrap function) and used by the OAuth provider factory.

type OpenAIDialect

type OpenAIDialect string

OpenAIDialect identifies the dialect of an OpenAI-compatible API. Different services have minor behavioural quirks; this enum lets us branch on them without breaking the core OpenAI flow.

const (
	DialectStandard OpenAIDialect = ""         // standard OpenAI
	DialectGemini   OpenAIDialect = "gemini"   // Google Gemini OpenAI compat
	DialectMistral  OpenAIDialect = "mistral"  // Mistral
	DialectGroq     OpenAIDialect = "groq"     // Groq
	DialectDeepSeek OpenAIDialect = "deepseek" // DeepSeek
	DialectOllama   OpenAIDialect = "ollama"   // Ollama local
)

type OpenAIOAuthResult

type OpenAIOAuthResult struct {
	APIKey              string
	APIKeyExchangeError string
	AccessToken         string
	RefreshToken        string
	ExpiresAt           time.Time
	AccountID           string
	PlanType            string
	AccountIsFedRAMP    bool
}

func StartOpenAIOAuthFlow

func StartOpenAIOAuthFlow(ctx context.Context, authURLOut chan<- string) (*OpenAIOAuthResult, error)

StartOpenAIOAuthFlow runs the Codex-style OpenAI browser OAuth flow and returns the ChatGPT OAuth tokens plus an API-key exchange result when the auth server makes one available.

type OpenAIProvider

type OpenAIProvider struct {
	// contains filtered or unexported fields
}

OpenAIProvider implements Provider for OpenAI-compatible APIs Works with: OpenAI, DeepSeek, Ollama, vLLM, LiteLLM, Azure OpenAI, etc.

func NewOpenAI

func NewOpenAI(cfg Config) *OpenAIProvider

NewOpenAI creates a Provider for OpenAI-compatible APIs.

func (*OpenAIProvider) Capabilities

func (p *OpenAIProvider) Capabilities() ProviderCapabilities

Capabilities implements CapabilityProvider for OpenAIProvider.

func (*OpenAIProvider) CreateStream

func (p *OpenAIProvider) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

func (*OpenAIProvider) ModelID

func (p *OpenAIProvider) ModelID() string

func (*OpenAIProvider) Name

func (p *OpenAIProvider) Name() string

type Params

type Params struct {
	Model     string
	MaxTokens int
	// MaxOutputTokensOverride temporarily overrides MaxTokens for output-limit
	// recovery paths. Zero preserves the provider's normal MaxTokens behavior.
	MaxOutputTokensOverride int
	System                  string                     // single system prompt block
	SystemBlocks            []prompt.SystemPromptBlock // ordered system prompt blocks with optional metadata
	Messages                []types.Message
	Tools                   []types.ToolDefinition
	ExtraToolSchemas        []types.ServerToolDefinition
	ToolChoice              *ToolChoice     // nil = auto (default)
	Thinking                *ThinkingConfig // nil = thinking disabled
	TaskBudget              *TaskBudget     // nil = API-side task budget disabled

	// Responses API specific fields
	Conversation       string // Session ID for conversation tracking
	PreviousResponseID string // Link to previous response for context continuity
	Truncation         string // "auto", "disabled", "required_or_error"

	// Prompt cache routing fields shared by Responses and compatible Chat APIs.
	PromptCacheKey string // Stable session lineage lowered to a user-level routing shard by capable providers
	UsePromptCache bool   // Mirror Codex CLI: keep prompt_cache_key enabled even after previous_response_id fallback
	// PromptCacheTTL is provider-selected. Empty preserves the provider's
	// default; "1h" requests the longest documented Anthropic-compatible TTL.
	PromptCacheTTL  string
	ReasoningEffort string // "low", "medium", "high" for reasoning models
	ServiceTier     ServiceTier

	// ContinuationLineage/Epoch are content-free stateless replay evidence.
	// A higher epoch with ContinuationReset proves an intentional context
	// replacement (for example compaction) without exposing message content.
	ContinuationLineage string
	ContinuationEpoch   uint64
	ContinuationReset   bool
	// contains filtered or unexported fields
}

Params holds LLM-agnostic request parameters

func (Params) JoinedSystemPrompt

func (p Params) JoinedSystemPrompt() string

JoinedSystemPrompt returns the stable string representation used by providers that do not have native multi-block system prompt support.

func (Params) SystemTextBlocks

func (p Params) SystemTextBlocks() []prompt.SystemPromptBlock

SystemTextBlocks resolves the system prompt fields into provider-ready text blocks. SystemBlocks take priority over the single System string.

func (Params) WithInternalControlScope

func (p Params) WithInternalControlScope(capability messagecontrol.Capability, scope messagecontrol.Scope) Params

WithInternalControlScope installs the runtime-owned authority fence used by provider adapters. Its capability type is internal to this module, so SDK callers cannot select a scope or turn arbitrary developer messages into privileged instructions.

type Provider

type Provider interface {
	// Name returns the provider identifier
	Name() string

	// CreateStream sends a streaming message request and returns events via channel
	CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

	// ModelID returns the default model for this provider
	ModelID() string
}

Provider is the abstraction for any LLM backend (Anthropic, OpenAI, Ollama, etc.)

func NewFromEnvWithOverrides

func NewFromEnvWithOverrides(providerOverride, modelOverride string) (Provider, error)

NewFromEnvWithOverrides creates a provider from environment variables, but allows explicit overrides for provider name and model. This avoids mutating global env with os.Setenv, which is not concurrency-safe.

When a CredentialStore is attached to the registry, provider factories can also look up credentials from the store (see registry_builtins.go).

func NewFromEnvWithRuntimeOverrides

func NewFromEnvWithRuntimeOverrides(providerOverride, modelOverride string, overrides RuntimeOverrides) (Provider, error)

NewFromEnvWithRuntimeOverrides resolves stored credentials, then applies the caller's typed invocation settings to the exact Config consumed by the provider factory. Responses WebSocket remains fail-closed unless Supported is explicitly supplied.

type ProviderCapabilities

type ProviderCapabilities struct {
	Thinking           bool
	ToolUse            bool
	CustomTools        CapabilitySupport
	ResponsesWebSocket CapabilitySupport
	ServiceTier        CapabilitySupport
	CacheControl       bool
	CacheRouting       CacheRoutingMode
	SystemParts        bool
	Vision             bool
	MaxContext         int // token limit; 0 = unknown
}

ProviderCapabilities describes the features a provider supports. Zero values mean "not supported" or "unknown".

type ProviderFactory

type ProviderFactory func(cfg Config, modelOverride string) (Provider, error)

ProviderFactory creates a Provider from a Config and optional model override. The context is provided for factories that need it (e.g. AWS credential chain).

type ProviderInfo

type ProviderInfo struct {
	// Name is the registered provider identifier (e.g. "anthropic", "openai").
	Name string

	// DisplayName is the human-readable name (e.g. "Anthropic", "OpenAI").
	DisplayName string

	// EnvKey is the primary environment variable for authentication (e.g. "ANTHROPIC_API_KEY").
	// Empty for providers that don't use an API key (e.g. "ollama").
	EnvKey string

	// Models lists the canonical model IDs available for this provider.
	Models []string

	// DefaultModel is the model used when none is specified.
	DefaultModel string

	// AuthMethods describes supported authentication methods.
	// Values: "api_key", "oauth_pkce", "device_code", "aws_credentials", "gcp_adc"
	AuthMethods []string

	// Hidden marks internal providers omitted from user-facing pickers.
	Hidden bool

	// Popularity is a sorting weight (higher = more popular, shown first).
	Popularity int

	// RequiresContext indicates the factory needs a context.Context (e.g. Bedrock, Vertex).
	RequiresContext bool

	// DefaultBaseURL is the default API base URL for this provider.
	// Empty means the provider does not have a well-known default or uses
	// the provider-specific SDK default.
	DefaultBaseURL string

	// APIStyles lists the wire protocols offered by a compatible aggregate
	// provider. Empty means the provider uses its native, fixed protocol.
	APIStyles []APIStyle

	// DefaultBaseURLs maps each compatible protocol to its default endpoint.
	// It is populated for named aggregate providers and empty for user-defined
	// gateways, which must supply their own Base URL.
	DefaultBaseURLs map[APIStyle]string

	// DynamicModels marks providers whose model catalog is discovered from the
	// configured endpoint after authentication.
	DynamicModels bool

	// UserDefined marks providers created through the "other gateway" flow.
	UserDefined bool
}

ProviderInfo describes a registered provider's metadata and capabilities.

func (ProviderInfo) BaseURLForStyle

func (info ProviderInfo) BaseURLForStyle(style APIStyle) string

BaseURLForStyle returns the protocol-specific default URL, falling back to the legacy single default for older provider definitions.

type ProviderRef

type ProviderRef struct {
	// contains filtered or unexported fields
}

ProviderRef is a thread-safe, swappable Provider reference. It implements the Provider and CapabilityProvider interfaces as a transparent proxy, so all existing code can receive a *ProviderRef anywhere a Provider is expected — zero call-site changes needed.

The provider can be atomically replaced at runtime via Swap(). All method calls are forwarded to the current underlying provider. Provider replacement takes effect between queries; an in-flight CreateStream call uses the provider snapshot taken at call time.

func NewProviderRef

func NewProviderRef(p Provider) *ProviderRef

NewProviderRef creates a ProviderRef wrapping the given initial provider.

func (*ProviderRef) APIFormat

func (r *ProviderRef) APIFormat() string

APIFormat exposes the snapshotted provider's wire protocol when available.

func (*ProviderRef) ApplyRequestPolicy

func (r *ProviderRef) ApplyRequestPolicy(params Params) Params

ApplyRequestPolicy projects process-wide request policy onto a parameter snapshot without replacing an explicit per-call value. Callers that must retain a Provider returned by Get can use this before invoking that exact provider, avoiding policy bypass during auxiliary generations.

func (*ProviderRef) Capabilities

func (r *ProviderRef) Capabilities() ProviderCapabilities

Capabilities delegates to the current provider if it implements CapabilityProvider, otherwise returns zero-value capabilities.

func (*ProviderRef) Close

func (r *ProviderRef) Close() error

Close releases the current provider transport. The application calls this only after engines and background consumers have drained.

func (*ProviderRef) CreateStream

func (r *ProviderRef) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

CreateStream delegates to the current provider. The provider is snapshotted at the start of the call, so a concurrent Swap() does not affect an in-flight stream.

func (*ProviderRef) Get

func (r *ProviderRef) Get() Provider

Get returns the current underlying provider.

func (*ProviderRef) ModelID

func (r *ProviderRef) ModelID() string

ModelID delegates to the current provider.

func (*ProviderRef) Name

func (r *ProviderRef) Name() string

Name delegates to the current provider.

func (*ProviderRef) OnChange

func (r *ProviderRef) OnChange(fn func(Provider))

OnChange registers a callback that fires after each Swap(). Callbacks run outside the lock and receive the new provider.

func (*ProviderRef) SetDebugObserver

func (r *ProviderRef) SetDebugObserver(observer DebugObserver)

SetDebugObserver installs or clears the observer used for complete LLM request/response snapshots. It is safe to call while requests are in flight; each request retains the observer that was active when it started.

func (*ProviderRef) SetServiceTier

func (r *ProviderRef) SetServiceTier(serviceTier ServiceTier)

SetServiceTier installs a process-wide request policy for calls routed through this reference. Explicit per-call values remain visible to validation, while omitted values inherit the pinned tier. This covers compaction, goal evaluation, web helpers, and subagents that share the ref.

func (*ProviderRef) Swap

func (r *ProviderRef) Swap(p Provider) Provider

Swap atomically replaces the underlying provider and notifies all registered onChange listeners. Returns the previous provider.

func (*ProviderRef) UnsupportedRequestFields

func (r *ProviderRef) UnsupportedRequestFields(model string) []string

type ProviderRegistry

type ProviderRegistry struct {
	// contains filtered or unexported fields
}

ProviderRegistry is a central registry of available providers and their factories. It maintains a catalog of all known models across all providers.

func DefaultRegistry

func DefaultRegistry() *ProviderRegistry

DefaultRegistry returns the singleton ProviderRegistry pre-populated with all built-in providers. The registry is lazily initialized on first call.

func NewProviderRegistry

func NewProviderRegistry() *ProviderRegistry

NewProviderRegistry creates an empty ProviderRegistry with a new ModelCatalog.

func (*ProviderRegistry) All

func (r *ProviderRegistry) All() []ProviderInfo

All returns every registered ProviderInfo, sorted by popularity (descending) then name.

func (*ProviderRegistry) ApplyModelOverrides

func (r *ProviderRegistry) ApplyModelOverrides(overrides ModelOverrides)

func (*ProviderRegistry) Available

func (r *ProviderRegistry) Available() []ProviderInfo

Available returns providers that can be selected for model use. Prefer ConnectionState when UI or commands need to distinguish configured, local, and setup-required providers.

func (*ProviderRegistry) Catalog

func (r *ProviderRegistry) Catalog() *ModelCatalog

Catalog returns the ModelCatalog associated with this registry.

func (*ProviderRegistry) ConnectionState

func (r *ProviderRegistry) ConnectionState(name string) ConnectionDetail

ConnectionState returns a typed connection status for a registered provider.

func (*ProviderRegistry) Create

func (r *ProviderRegistry) Create(name string, cfg Config, modelOverride string) (Provider, error)

Create instantiates a Provider by name using the registered factory.

func (*ProviderRegistry) CredentialStoreRef

func (r *ProviderRegistry) CredentialStoreRef() *CredentialStore

CredentialStore returns the attached CredentialStore, or nil.

func (*ProviderRegistry) DiscoverCompatibleModels

func (r *ProviderRegistry) DiscoverCompatibleModels(ctx context.Context, input CompatibleModelRequest) ([]ModelInfo, error)

DiscoverCompatibleModels lists models from a compatible endpoint and fills missing metadata from normalized matches in the built-in catalog.

func (*ProviderRegistry) Get

func (r *ProviderRegistry) Get(name string) (ProviderInfo, bool)

Get returns the ProviderInfo for a given name.

func (*ProviderRegistry) Names

func (r *ProviderRegistry) Names() []string

Names returns all registered provider names, sorted alphabetically.

func (*ProviderRegistry) NextUserProviderName

func (r *ProviderRegistry) NextUserProviderName(displayName, baseURL string) string

NextUserProviderName creates a collision-free canonical identifier. The display name remains independent and can contain the user's original text.

func (*ProviderRegistry) OAuthHookRef

func (r *ProviderRegistry) OAuthHookRef() OAuthHook

OAuthHookRef returns the attached OAuthHook, or nil.

func (*ProviderRegistry) Register

func (r *ProviderRegistry) Register(info ProviderInfo, factory ProviderFactory)

Register adds a provider with its factory to the registry.

func (*ProviderRegistry) RegisterCompatibleProvider

func (r *ProviderRegistry) RegisterCompatibleProvider(def CompatibleProviderDefinition)

RegisterCompatibleProvider installs a shared OpenAI/Anthropic-compatible factory. Named cloud plans and user-created gateways use this same path.

func (*ProviderRegistry) ReplaceProviderModels

func (r *ProviderRegistry) ReplaceProviderModels(providerName string, models []ModelInfo)

ReplaceProviderModels refreshes the shared catalog and the corresponding ProviderInfo snapshot as one registry-level operation.

func (*ProviderRegistry) SetCredentialStore

func (r *ProviderRegistry) SetCredentialStore(cs *CredentialStore)

SetCredentialStore attaches a CredentialStore to the registry. When set, Available() also checks the store.

func (*ProviderRegistry) SetOAuthHook

func (r *ProviderRegistry) SetOAuthHook(hook OAuthHook)

SetOAuthHook attaches an OAuthHook to the registry. The hook is used by the OAuth provider factory for token management.

func (*ProviderRegistry) UnregisterUserProvider

func (r *ProviderRegistry) UnregisterUserProvider(name string) bool

UnregisterUserProvider removes a user-created provider and its discovered models. Built-in providers cannot be removed through this method.

func (*ProviderRegistry) Visible

func (r *ProviderRegistry) Visible() []ProviderInfo

Visible returns registered ProviderInfo entries intended for user-facing provider selection, sorted by popularity (descending) then name.

func (*ProviderRegistry) VisibleNames

func (r *ProviderRegistry) VisibleNames() []string

VisibleNames returns non-hidden registered provider names, sorted alphabetically.

type ResponsesProvider

type ResponsesProvider struct {
	// contains filtered or unexported fields
}

ResponsesProvider implements Provider for the OpenAI Responses API (/v1/responses). It is stateless. Every request disables provider storage and replays the complete committed input history. OpenAI reasoning items are round-tripped as opaque encrypted items; response IDs remain observable evidence but are never treated as retrievable state while store=false.

func NewResponses

func NewResponses(cfg Config) *ResponsesProvider

NewResponses creates a Provider for the OpenAI Responses API.

func (*ResponsesProvider) APIFormat

func (p *ResponsesProvider) APIFormat() string

APIFormat returns the API protocol used by this provider.

func (*ResponsesProvider) ApplyCredentialConfig

func (p *ResponsesProvider) ApplyCredentialConfig(cfg Config)

ApplyCredentialConfig updates request authentication and routing after an OAuth refresh without rebuilding the provider wrapper.

func (*ResponsesProvider) Capabilities

func (p *ResponsesProvider) Capabilities() ProviderCapabilities

Capabilities implements CapabilityProvider for ResponsesProvider.

func (*ResponsesProvider) Close

func (p *ResponsesProvider) Close() error

Close releases persistent Responses sockets. It is intentionally optional on the Provider interface so existing providers remain source-compatible.

func (*ResponsesProvider) CreateStream

func (p *ResponsesProvider) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

CreateStream selects WebSocket mode only for an explicitly verified public Responses endpoint and a loop-private continuation lineage. Every other profile remains on the stable HTTP/SSE transport.

func (*ResponsesProvider) ModelID

func (p *ResponsesProvider) ModelID() string

func (*ResponsesProvider) Name

func (p *ResponsesProvider) Name() string

func (*ResponsesProvider) TryFallbackTransport

func (p *ResponsesProvider) TryFallbackTransport() (from, to string, activated bool)

TryFallbackTransport permanently disables Responses WebSocket transport for this provider instance and clears its connection-local continuation state. A fresh HTTP request will replay the complete committed local history.

func (*ResponsesProvider) UnsupportedRequestFields

func (p *ResponsesProvider) UnsupportedRequestFields(model string) []string

type ResponsesSemantics

type ResponsesSemantics string

ResponsesSemantics is the capability profile for a Responses endpoint. It deliberately does not infer authority from a hostname. Auto exists for direct library compatibility; production provider factories set an explicit profile.

const (
	ResponsesSemanticsAuto         ResponsesSemantics = ""
	ResponsesSemanticsOpenAIPublic ResponsesSemantics = "openai_public"
	ResponsesSemanticsOpenAICodex  ResponsesSemantics = "openai_codex"
	ResponsesSemanticsDeepSeek     ResponsesSemantics = "deepseek"
	ResponsesSemanticsCompatible   ResponsesSemantics = "generic_compatible"
)

type RetryConfig

type RetryConfig struct {
	// MaxAttempts is the total HTTP request-attempt budget used before a
	// response stream is established. The default is five (the initial request
	// plus four retries), matching Codex CLI's request_max_retries default.
	MaxAttempts int
	// StreamMaxAttempts is the independent response-stream budget. The default
	// is six (the initial stream plus five reconnects), matching Codex CLI's
	// stream_max_retries default. Keeping the budgets separate prevents a few
	// connection failures from consuming all stream-recovery capacity.
	StreamMaxAttempts int
	BaseDelay         time.Duration // default 200 ms
	MaxDelay          time.Duration // default 3.2 s for the bounded default budgets
	// OnRetry is called before each retry attempt. nil = no logging.
	// attempt is the one-based failed raw attempt that triggered the retry.
	OnRetry func(attempt, maxRetries int, delay time.Duration, err error)
	// OnAuthError is called when a 401 Unauthorized error is encountered.
	// If it returns true, credentials were refreshed and one retry is allowed.
	// If nil or it returns false, the 401 is surfaced immediately (fail fast).
	OnAuthError func() bool
	// contains filtered or unexported fields
}

RetryConfig controls retry behaviour.

func DefaultRetryConfig

func DefaultRetryConfig() RetryConfig

DefaultRetryConfig returns the interactive LLM retry policy.

type RetryEvent

type RetryEvent struct {
	Attempt    int
	MaxRetries int
	Delay      time.Duration
	Err        error
	Kind       string
	// DroppedField is an implementation-owned allowlisted request field removed
	// by a compatible gateway fallback. It never contains provider text.
	DroppedField string
}

RetryEvent describes a transient provider failure immediately before the provider waits and tries the request again.

type RetryProvider

type RetryProvider struct {
	// contains filtered or unexported fields
}

RetryProvider is a decorator that wraps any Provider and adds retry logic. It implements the Provider interface and is transparent to callers.

func NewRetryProvider

func NewRetryProvider(inner Provider, cfg RetryConfig) *RetryProvider

NewRetryProvider creates a RetryProvider wrapping inner with the given config. Use DefaultRetryConfig() to get sensible defaults.

func (*RetryProvider) APIFormat

func (r *RetryProvider) APIFormat() string

APIFormat exposes the wrapped OpenAI-family wire protocol for diagnostics.

func (*RetryProvider) Capabilities

func (r *RetryProvider) Capabilities() ProviderCapabilities

Capabilities implements CapabilityProvider by delegating to the inner provider if it also implements CapabilityProvider. Returns zero-value if not supported.

func (*RetryProvider) Close

func (r *RetryProvider) Close() error

Close releases a persistent transport owned by the wrapped provider.

func (*RetryProvider) CreateStream

func (r *RetryProvider) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

CreateStream calls the inner provider's CreateStream, retrying on transient errors according to the RetryConfig.

Retry strategy:

  • At most MaxAttempts HTTP calls while establishing this stream.
  • Exponential backoff with ±10% jitter and Retry-After support.
  • Permanent 4xx, context, quota, billing, and model errors fail fast.
  • A 401 can refresh credentials at most once, then retries without delay.
  • Context cancellation aborts the retry loop immediately.

If CreateStream succeeds (returns a channel), the channel is returned directly without wrapping — stream-level EventErrors are forwarded as-is to the caller.

func (*RetryProvider) ModelID

func (r *RetryProvider) ModelID() string

ModelID delegates to the wrapped provider.

func (*RetryProvider) Name

func (r *RetryProvider) Name() string

Name delegates to the wrapped provider.

func (*RetryProvider) UnsupportedRequestFields

func (r *RetryProvider) UnsupportedRequestFields(model string) []string

type RuntimeOverrides

type RuntimeOverrides struct {
	APIFormat          string
	ResponsesWebSocket CapabilitySupport
}

RuntimeOverrides contains invocation-scoped wire choices that must cross the credential-resolution boundary without being inferred from environment or endpoint location. Zero values preserve the ordinary provider defaults.

type ServiceTier

type ServiceTier string

ServiceTier is the provider scheduling class requested for one generation. An empty value leaves scheduling unspecified. Contract-bound runs currently admit only ServiceTierDefault so an endpoint cannot silently choose auto, flex, or priority scheduling.

const ServiceTierDefault ServiceTier = "default"

type StreamIdleTimeoutError

type StreamIdleTimeoutError struct {
	Phase   streamWatchdogPhase
	IdleFor time.Duration
}

StreamIdleTimeoutError is returned only when a live streaming response has produced no complete stream event for its deadline. It is distinct from context cancellation and from a request-wide deadline, and is safe for the query loop to replay from its last committed response boundary.

func (*StreamIdleTimeoutError) Error

func (e *StreamIdleTimeoutError) Error() string

func (*StreamIdleTimeoutError) Temporary

func (e *StreamIdleTimeoutError) Temporary() bool

func (*StreamIdleTimeoutError) Timeout

func (e *StreamIdleTimeoutError) Timeout() bool

type StreamTransportFallback

type StreamTransportFallback interface {
	TryFallbackTransport() (from, to string, activated bool)
}

StreamTransportFallback is implemented by providers that can permanently switch a session away from a failed preferred streaming transport.

type TaskBudget

type TaskBudget struct {
	Total     int
	Remaining *int
}

TaskBudget controls API-side task output budgeting across compaction boundaries. Remaining is omitted until the loop has compacted at least once.

type ThinkingConfig

type ThinkingConfig struct {
	Enabled      bool
	BudgetTokens int // token budget for thinking; 0 means use model default (1024 minimum required by API)
}

ThinkingConfig controls extended thinking for a request

type TokenCountingProvider

type TokenCountingProvider interface {
	CountTokens(ctx context.Context, content string) (int, error)
}

TokenCountingProvider is an optional precise text-token counter. Read uses it only near configured limits and falls back to its local tokenizer when a provider does not expose the API or the request fails.

type ToolChoice

type ToolChoice struct {
	Type string // "auto", "any", "tool"
	Name string // only used when Type == "tool"
}

ToolChoice controls which tool (if any) the model must call. Type can be "auto" (default), "any" (must call some tool), or "tool" (must call specific tool).

type UnconfiguredProvider

type UnconfiguredProvider struct {
	// contains filtered or unexported fields
}

UnconfiguredProvider lets the terminal UI start before credentials exist. Actual model calls fail with an actionable message instead of blocking setup workflows such as /model and /config.

func NewUnconfiguredProvider

func NewUnconfiguredProvider(name, model, envVar, message string) *UnconfiguredProvider

func (*UnconfiguredProvider) Capabilities

func (p *UnconfiguredProvider) Capabilities() ProviderCapabilities

func (*UnconfiguredProvider) CreateStream

func (p *UnconfiguredProvider) CreateStream(context.Context, Params) (<-chan types.StreamEvent, error)

func (*UnconfiguredProvider) ModelID

func (p *UnconfiguredProvider) ModelID() string

func (*UnconfiguredProvider) Name

func (p *UnconfiguredProvider) Name() string

type VertexConfig

type VertexConfig struct {
	// ProjectID is the GCP project ID (GOOGLE_CLOUD_PROJECT / ANTHROPIC_VERTEX_PROJECT_ID).
	ProjectID string

	// Region is the GCP region (e.g. "us-east5"). Defaults to GOOGLE_CLOUD_REGION /
	// CLOUD_ML_REGION / ANTHROPIC_VERTEX_REGION env vars, falls back to "us-east5".
	Region string

	// Model is the Vertex model ID (e.g. "claude-sonnet-4-20250514").
	// Vertex uses the same short model names as the direct API (no "anthropic." prefix).
	Model string

	// MaxTokens overrides the default max output tokens.
	MaxTokens int

	// Timeout in seconds for requests.
	Timeout int

	// BaseURL overrides only the Vertex transport endpoint. Authentication and
	// request semantics remain Vertex-native.
	BaseURL string
}

VertexConfig holds Google Cloud configuration for the Vertex AI provider.

func VertexConfigFromEnv

func VertexConfigFromEnv() VertexConfig

VertexConfigFromEnv reads Vertex AI configuration from environment variables.

Variables read:

  • GOOGLE_CLOUD_PROJECT / ANTHROPIC_VERTEX_PROJECT_ID — GCP project ID
  • GOOGLE_CLOUD_REGION / CLOUD_ML_REGION / ANTHROPIC_VERTEX_REGION — GCP region
  • VERTEX_MODEL — model ID override
  • ANTHROPIC_VERTEX_BASE_URL — Vertex transport endpoint override
  • GOOGLE_APPLICATION_CREDENTIALS — path to service account JSON (used by ADC automatically)

type VertexProvider

type VertexProvider struct {
	// contains filtered or unexported fields
}

VertexProvider wraps the Anthropic SDK with Google Vertex AI transport.

func NewVertex

func NewVertex(ctx context.Context, cfg VertexConfig) (*VertexProvider, error)

NewVertex creates a Provider backed by Google Cloud Vertex AI.

Authentication uses Application Default Credentials (ADC):

  1. GOOGLE_APPLICATION_CREDENTIALS env → service account JSON
  2. gcloud application-default credentials
  3. Workload Identity (GKE / Cloud Run)

ctx is forwarded to vertex.WithGoogleAuth for credential initialization.

func (*VertexProvider) Capabilities

func (p *VertexProvider) Capabilities() ProviderCapabilities

Capabilities implements CapabilityProvider.

func (*VertexProvider) CreateStream

func (p *VertexProvider) CreateStream(ctx context.Context, params Params) (<-chan types.StreamEvent, error)

CreateStream uses the shared Anthropic stream implementation with our Vertex client.

func (*VertexProvider) ModelID

func (p *VertexProvider) ModelID() string

func (*VertexProvider) Name

func (p *VertexProvider) Name() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL