router

package
v0.0.0-...-218c81d Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 3, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package router picks which agent, model and effort to delegate a task to, and learns from how past runs went. It is pure Go: no model runs here.

  • Arms are chain links ("cli:model[:effort]") from a pool listed cheapest → strongest; an arm's cost rank is its place in the pool.
  • A task gets a type (report.TaskType) and a tier (simple, medium, complex: tier.go, one table of rules).
  • Every past run is a success, a failure or no signal (evidence.go), and each (arm, type, tier) keeps a recency-weighted Beta posterior with a fallback to coarser levels when it has few runs (posterior.go).
  • The choice is Thompson sampling with a cost penalty: draw a success rate from each arm's posterior and take the cheapest arm whose draw is within Margin of the best one. Uncertain arms sometimes draw high, so they get tried (exploration); as runs pile up the posteriors narrow and exploration fades on its own.
  • Optionally an external command classifies the task (external.go).

Index

Constants

View Source
const (
	PriorStrength = 3.0 // prior(tier, rank) × lift → (arm, tier)
	TierStrength  = 5.0 // (arm, tier) → (arm, type, tier), at most

)

Strengths of each level, in runs' worth of evidence: how much a cell with no runs of its own leans on the level above it.

View Source
const (
	Simple  = "simple"
	Medium  = "medium"
	Complex = "complex"
)

Tiers: how big a task looks. The router learns separately per tier, and its priors expect cheap arms to do well at simple tasks and strong arms at complex ones.

View Source
const ChecksFile = "router-checks.json"

ChecksFile is the cache's name inside runs.Home().

View Source
const DefaultExternalTimeout = 5 * time.Second

DefaultExternalTimeout is how long the external router may take.

View Source
const HalfLife = 30 * 24 * time.Hour

HalfLife: a run this old weighs half as much as one from today.

View Source
const Margin = 0.08

Margin: an arm whose sampled success is within this of the best one is as good, and the cheapest of those wins.

Variables

Tiers in order.

Functions

func ArmKey

func ArmKey(cli, model, effort string) string

ArmKey is the arm of a run: "cli", "cli:model", "cli:model:effort" or "cli::effort", the way panal delegate writes a chain link.

func Prior

func Prior(tier string, rank, n int) float64

Prior is the success rate expected of an arm before any run, by tier and cost rank (rank 0 = cheapest of n arms, n-1 = strongest). Cheap arms are expected to do well at simple tasks; complex tasks are expected to need the strong ones:

simple   0.80 → 0.85
medium   0.60 → 0.80
complex  0.35 → 0.75

linearly from the cheapest arm to the strongest.

func Summarize

func Summarize(rs []history.Run, since time.Time, checks *Checks, now time.Time) report.RouterSummary

Summarize is the report's Router section: the auto decisions since a date (one per `panal delegate -c auto`, found by its first link), how often the first choice succeeded, and how many picks explored.

func Weight

func Weight(t, now time.Time) float64

Weight of a run that ended at t, seen at now: 0.5^(age/HalfLife).

Types

type Beta

type Beta struct{ A, B float64 }

Beta is a Beta(Alpha, Beta) posterior over an arm's success rate.

func (Beta) Mean

func (b Beta) Mean() float64

Mean of the distribution.

func (Beta) Strength

func (b Beta) Strength() float64

Strength is A+B: how many runs' worth of evidence it holds.

type Check

type Check struct {
	Violated    bool      `json:"violated,omitempty"`
	TestsSeen   bool      `json:"tests_seen,omitempty"`
	TestsPassed bool      `json:"tests_passed,omitempty"`
	At          time.Time `json:"at"`
}

Check is the part of an observation that costs git or log reads: whether the run broke its task and how its tests went. It is final once the run has ended, so it is cached in ~/.panal/router-checks.json.

func CheckRun

func CheckRun(r history.Run, now time.Time) Check

CheckRun checks a run for real: its changes against the task's rules (git, read-only) and the last tests in its log.

type Checks

type Checks struct {
	Path string // "" = do not persist

	// Run checks a run (tests replace it); default CheckRun.
	Run func(history.Run, time.Time) Check
	// contains filtered or unexported fields
}

Checks caches the checks of finished runs, by run key.

func LoadChecks

func LoadChecks(path string) *Checks

LoadChecks reads the cache (a missing or broken file is an empty cache).

func (*Checks) Get

func (c *Checks) Get(r history.Run, now time.Time) Check

Get returns the check of a finished run, computing it if needed. Only done runs need one: every other status decides the outcome on its own.

func (*Checks) Put

func (c *Checks) Put(key string, v Check)

Put stores a check.

func (*Checks) Save

func (c *Checks) Save() error

Save writes the cache if it changed and has a path.

type Counts

type Counts struct {
	S, F   float64 // weighted
	OK, N  int     // raw: successes and runs with a signal
	Recent [2]int  // raw, last 30 days: successes, runs
}

Counts are recency-weighted successes and failures, plus the raw number of runs behind them (for the explanation).

type Decision

type Decision struct {
	Type, Tier string
	Score      int   // the tier rules' points
	Hits       []Hit // the tier rules that applied
	Estimates  []Estimate
	Draws      map[string]float64 // the Thompson draw of each arm
	Chain      []string           // best first; the rest is the fallback
	Favorite   string             // the first arm by posterior mean (no exploration)
	Explored   bool               // Chain[0] is not the favorite: this pick explores
	ExploreP   float64            // how often a draw picks something other than the favorite
	External   string             // what the external router did ("" = not configured)
	Picked     bool               // the external router picked Chain[0]
	Why        string             // one line, in English
}

Decision is the router's answer.

func Decide

func Decide(q Request) (Decision, error)

Decide routes a task.

func (Decision) Estimate

func (d Decision) Estimate(arm string) (Estimate, bool)

Estimate returns the estimate of arm, if it is in the decision.

type Estimate

type Estimate struct {
	Arm      string
	Rank     int
	Prior    float64 // prior(tier, rank)
	Lift     float64 // the arm's record against the prior over all its runs
	Base     float64 // the prior with its odds multiplied by Lift
	Arm1     Counts  // the arm, every type and tier
	ArmTier  Counts  // the arm, this tier
	Cell     Counts  // the arm, this type and tier
	TierPost Beta    // (arm, tier)
	Post     Beta    // (arm, type, tier): what is sampled
}

Estimate is the posterior of one arm for one task, with how it was built.

type Evidence

type Evidence struct {
	Key     string // run key (<ID>-<agent>)
	Arm     string
	CLI     string
	Type    string
	Tier    string
	At      time.Time // when it ended (else started)
	Outcome Outcome
}

Evidence is one past run, reduced to what the router learns from.

func Gather

func Gather(rs []history.Run, checks *Checks, now time.Time) []Evidence

Gather turns past runs into evidence. Runs without a recorded type or tier (older than the router, or not delegated by panal) are classified now from their task.

func ResolveArms

func ResolveArms(ev []Evidence, pool []string) []Evidence

ResolveArms maps each run to the pool arm it is evidence for: its own arm when that is in the pool; otherwise, when the pool has its CLI bare ("codex": the CLI's own default model and effort), that bare arm, since a run with a model the pool doesn't name is the closest thing to it. Other runs keep their arm and only show in the stats.

type External

type External struct {
	Command string
	Timeout time.Duration // 0 = DefaultExternalTimeout
	Env     []string      // added to the environment (tests)
}

External is an optional classifier the user plugs in with `router = <command>` in panal.conf (or PANAL_ROUTER): a program (and its arguments, split on spaces; no shell) that gets the task as JSON on stdin and answers JSON on stdout within Timeout:

in:  {"task": "...", "dir": "...", "read_only": false,
      "type": "fix", "tier": "medium", "score": 0,
      "pool": ["codex:gpt-6-luna:low", "agy"]}
out: {"tier": "simple|medium|complex", "arm": "optional, one of pool"}

"tier" replaces the built-in tier; "arm", if given and in the pool, goes first in the chain. On any error (it can't start, exits non-zero, takes too long, prints something that isn't that JSON) panal uses the built-in router and says so in the explanation. It can be anything: a script with your own rules, a small decision model, a local Ollama model.

func (*External) Ask

func (x *External) Ask(q Request, d Decision) (ExternalAnswer, error)

Ask runs the external router for a request and the built-in decision so far.

type ExternalAnswer

type ExternalAnswer struct {
	Tier string `json:"tier"`
	Arm  string `json:"arm,omitempty"`
}

ExternalAnswer is what it prints on stdout.

type ExternalInput

type ExternalInput struct {
	Task     string   `json:"task"`
	Dir      string   `json:"dir,omitempty"`
	ReadOnly bool     `json:"read_only"`
	Type     string   `json:"type"`
	Tier     string   `json:"tier"`
	Score    int      `json:"score"`
	Pool     []string `json:"pool"`
}

ExternalInput is what the external router reads on stdin.

type Features

type Features struct {
	Type     string // report.TaskType: review, fix, tests, refactor, docs, feature, other
	Length   int    // characters of the task's goal (report.Goal)
	Files    int    // files the task allows to modify (changes.Allowed)
	Dirs     int    // directories it allows to modify
	Steps    int    // numbered or bulleted lines
	ReadOnly bool
	// contains filtered or unexported fields
}

Features are what the router looks at in a task. Nothing here runs a model: it is the task text, the read-only flag and the task's own rules.

func Extract

func Extract(task string, readOnly bool) Features

Extract computes the features of a task.

type Hit

type Hit struct {
	Rule   string
	Points int
}

Hit is a rule that applied to a task.

func Classify

func Classify(f Features) (tier string, score int, hits []Hit)

Classify gives the tier of a task, with the rules that applied.

type Observation

type Observation struct {
	Status      runs.Status
	Violated    bool   // it broke the task's rules (internal/changes)
	Rating      string // the user's rating (internal/feedback)
	TestsSeen   bool   // tests ran in its activity (internal/activity)
	TestsPassed bool   // and the last ones passed
}

Observation is everything known about how a run went.

func Observe

func Observe(r history.Run, checks *Checks, now time.Time) Observation

Observe builds the observation of a run.

type Outcome

type Outcome int

Outcome of a past run, as the router learns from it.

const (
	NoSignal Outcome = iota // out of quota, skipped, interrupted, still running: says nothing about the arm
	Success
	Failure
)

func OutcomeOf

func OutcomeOf(o Observation) Outcome

OutcomeOf maps an observation to an outcome.

The user's rating wins: good is a success and bad a failure, whatever the status said (they looked at the work), except for runs where the arm never worked on the task (running, out of quota, skipped). Without a rating:

Success: status done, no rule broken and, if tests were seen in its activity, the last ones passed.

Failure: failed, timeout, no permission, a broken rule, or done with its last tests failing.

No signal: out of quota, skipped, interrupted, running, or anything else.

func (Outcome) String

func (o Outcome) String() string

type Request

type Request struct {
	Task     string
	Dir      string
	ReadOnly bool
	// Pool is the arms, cheapest first, as chain links ("cli:model[:effort]").
	Pool []string
	// Evidence is what the past runs say (Gather).
	Evidence []Evidence
	// OutOfQuota: CLIs known to be out of quota right now. Their arms go to
	// the end of the chain.
	OutOfQuota map[string]bool
	Now        time.Time
	Rand       *rand.Rand // nil = seeded from the clock
	External   *External  // nil = only the built-in router
}

Request is a task to route.

type Table

type Table = table

Table is the learned evidence (see Learn).

func Learn

func Learn(ev []Evidence, now time.Time, rank func(arm string) (int, int)) *Table

Learn builds the table from evidence (already resolved to pool arms with ResolveArms); rank gives an arm's cost rank and the pool size.

func (*Table) Estimate

func (t *Table) Estimate(arm string, rank, n int, typ, tier string) Estimate

Estimate is the posterior of arm (cost rank of n) for a task of type and tier.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL