Documentation
¶
Overview ¶
Package router picks which agent, model and effort to delegate a task to, and learns from how past runs went. It is pure Go: no model runs here.
- Arms are chain links ("cli:model[:effort]") from a pool listed cheapest → strongest; an arm's cost rank is its place in the pool.
- A task gets a type (report.TaskType) and a tier (simple, medium, complex: tier.go, one table of rules).
- Every past run is a success, a failure or no signal (evidence.go), and each (arm, type, tier) keeps a recency-weighted Beta posterior with a fallback to coarser levels when it has few runs (posterior.go).
- The choice is Thompson sampling with a cost penalty: draw a success rate from each arm's posterior and take the cheapest arm whose draw is within Margin of the best one. Uncertain arms sometimes draw high, so they get tried (exploration); as runs pile up the posteriors narrow and exploration fades on its own.
- Optionally an external command classifies the task (external.go).
Index ¶
- Constants
- Variables
- func ArmKey(cli, model, effort string) string
- func Prior(tier string, rank, n int) float64
- func Summarize(rs []history.Run, since time.Time, checks *Checks, now time.Time) report.RouterSummary
- func Weight(t, now time.Time) float64
- type Beta
- type Check
- type Checks
- type Counts
- type Decision
- type Estimate
- type Evidence
- type External
- type ExternalAnswer
- type ExternalInput
- type Features
- type Hit
- type Observation
- type Outcome
- type Request
- type Table
Constants ¶
const ( PriorStrength = 3.0 // prior(tier, rank) × lift → (arm, tier) TierStrength = 5.0 // (arm, tier) → (arm, type, tier), at most )
Strengths of each level, in runs' worth of evidence: how much a cell with no runs of its own leans on the level above it.
const ( Simple = "simple" Medium = "medium" Complex = "complex" )
Tiers: how big a task looks. The router learns separately per tier, and its priors expect cheap arms to do well at simple tasks and strong arms at complex ones.
const ChecksFile = "router-checks.json"
ChecksFile is the cache's name inside runs.Home().
const DefaultExternalTimeout = 5 * time.Second
DefaultExternalTimeout is how long the external router may take.
const HalfLife = 30 * 24 * time.Hour
HalfLife: a run this old weighs half as much as one from today.
const Margin = 0.08
Margin: an arm whose sampled success is within this of the best one is as good, and the cheapest of those wins.
Variables ¶
var Tiers = []string{Simple, Medium, Complex}
Tiers in order.
Functions ¶
func ArmKey ¶
ArmKey is the arm of a run: "cli", "cli:model", "cli:model:effort" or "cli::effort", the way panal delegate writes a chain link.
func Prior ¶
Prior is the success rate expected of an arm before any run, by tier and cost rank (rank 0 = cheapest of n arms, n-1 = strongest). Cheap arms are expected to do well at simple tasks; complex tasks are expected to need the strong ones:
simple 0.80 → 0.85 medium 0.60 → 0.80 complex 0.35 → 0.75
linearly from the cheapest arm to the strongest.
func Summarize ¶
func Summarize(rs []history.Run, since time.Time, checks *Checks, now time.Time) report.RouterSummary
Summarize is the report's Router section: the auto decisions since a date (one per `panal delegate -c auto`, found by its first link), how often the first choice succeeded, and how many picks explored.
Types ¶
type Beta ¶
type Beta struct{ A, B float64 }
Beta is a Beta(Alpha, Beta) posterior over an arm's success rate.
type Check ¶
type Check struct {
Violated bool `json:"violated,omitempty"`
TestsSeen bool `json:"tests_seen,omitempty"`
TestsPassed bool `json:"tests_passed,omitempty"`
At time.Time `json:"at"`
}
Check is the part of an observation that costs git or log reads: whether the run broke its task and how its tests went. It is final once the run has ended, so it is cached in ~/.panal/router-checks.json.
type Checks ¶
type Checks struct {
Path string // "" = do not persist
// Run checks a run (tests replace it); default CheckRun.
Run func(history.Run, time.Time) Check
// contains filtered or unexported fields
}
Checks caches the checks of finished runs, by run key.
func LoadChecks ¶
LoadChecks reads the cache (a missing or broken file is an empty cache).
type Counts ¶
type Counts struct {
S, F float64 // weighted
OK, N int // raw: successes and runs with a signal
Recent [2]int // raw, last 30 days: successes, runs
}
Counts are recency-weighted successes and failures, plus the raw number of runs behind them (for the explanation).
type Decision ¶
type Decision struct {
Type, Tier string
Score int // the tier rules' points
Hits []Hit // the tier rules that applied
Estimates []Estimate
Draws map[string]float64 // the Thompson draw of each arm
Chain []string // best first; the rest is the fallback
Favorite string // the first arm by posterior mean (no exploration)
Explored bool // Chain[0] is not the favorite: this pick explores
ExploreP float64 // how often a draw picks something other than the favorite
External string // what the external router did ("" = not configured)
Picked bool // the external router picked Chain[0]
Why string // one line, in English
}
Decision is the router's answer.
type Estimate ¶
type Estimate struct {
Arm string
Rank int
Prior float64 // prior(tier, rank)
Lift float64 // the arm's record against the prior over all its runs
Base float64 // the prior with its odds multiplied by Lift
Arm1 Counts // the arm, every type and tier
ArmTier Counts // the arm, this tier
Cell Counts // the arm, this type and tier
TierPost Beta // (arm, tier)
Post Beta // (arm, type, tier): what is sampled
}
Estimate is the posterior of one arm for one task, with how it was built.
type Evidence ¶
type Evidence struct {
Key string // run key (<ID>-<agent>)
Arm string
CLI string
Type string
Tier string
At time.Time // when it ended (else started)
Outcome Outcome
}
Evidence is one past run, reduced to what the router learns from.
func Gather ¶
Gather turns past runs into evidence. Runs without a recorded type or tier (older than the router, or not delegated by panal) are classified now from their task.
func ResolveArms ¶
ResolveArms maps each run to the pool arm it is evidence for: its own arm when that is in the pool; otherwise, when the pool has its CLI bare ("codex": the CLI's own default model and effort), that bare arm, since a run with a model the pool doesn't name is the closest thing to it. Other runs keep their arm and only show in the stats.
type External ¶
type External struct {
Command string
Timeout time.Duration // 0 = DefaultExternalTimeout
Env []string // added to the environment (tests)
}
External is an optional classifier the user plugs in with `router = <command>` in panal.conf (or PANAL_ROUTER): a program (and its arguments, split on spaces; no shell) that gets the task as JSON on stdin and answers JSON on stdout within Timeout:
in: {"task": "...", "dir": "...", "read_only": false,
"type": "fix", "tier": "medium", "score": 0,
"pool": ["codex:gpt-6-luna:low", "agy"]}
out: {"tier": "simple|medium|complex", "arm": "optional, one of pool"}
"tier" replaces the built-in tier; "arm", if given and in the pool, goes first in the chain. On any error (it can't start, exits non-zero, takes too long, prints something that isn't that JSON) panal uses the built-in router and says so in the explanation. It can be anything: a script with your own rules, a small decision model, a local Ollama model.
type ExternalAnswer ¶
ExternalAnswer is what it prints on stdout.
type ExternalInput ¶
type ExternalInput struct {
Task string `json:"task"`
Dir string `json:"dir,omitempty"`
ReadOnly bool `json:"read_only"`
Type string `json:"type"`
Tier string `json:"tier"`
Score int `json:"score"`
Pool []string `json:"pool"`
}
ExternalInput is what the external router reads on stdin.
type Features ¶
type Features struct {
Type string // report.TaskType: review, fix, tests, refactor, docs, feature, other
Length int // characters of the task's goal (report.Goal)
Files int // files the task allows to modify (changes.Allowed)
Dirs int // directories it allows to modify
Steps int // numbered or bulleted lines
ReadOnly bool
// contains filtered or unexported fields
}
Features are what the router looks at in a task. Nothing here runs a model: it is the task text, the read-only flag and the task's own rules.
type Observation ¶
type Observation struct {
Status runs.Status
Violated bool // it broke the task's rules (internal/changes)
Rating string // the user's rating (internal/feedback)
TestsSeen bool // tests ran in its activity (internal/activity)
TestsPassed bool // and the last ones passed
}
Observation is everything known about how a run went.
type Outcome ¶
type Outcome int
Outcome of a past run, as the router learns from it.
func OutcomeOf ¶
func OutcomeOf(o Observation) Outcome
OutcomeOf maps an observation to an outcome.
The user's rating wins: good is a success and bad a failure, whatever the status said (they looked at the work), except for runs where the arm never worked on the task (running, out of quota, skipped). Without a rating:
Success: status done, no rule broken and, if tests were seen in its activity, the last ones passed.
Failure: failed, timeout, no permission, a broken rule, or done with its last tests failing.
No signal: out of quota, skipped, interrupted, running, or anything else.
type Request ¶
type Request struct {
Task string
Dir string
ReadOnly bool
// Pool is the arms, cheapest first, as chain links ("cli:model[:effort]").
Pool []string
// Evidence is what the past runs say (Gather).
Evidence []Evidence
// OutOfQuota: CLIs known to be out of quota right now. Their arms go to
// the end of the chain.
OutOfQuota map[string]bool
Now time.Time
Rand *rand.Rand // nil = seeded from the clock
External *External // nil = only the built-in router
}
Request is a task to route.