Documentation
¶
Overview ¶
Package pruning removes stale material from the model context without a model call (duplicate tool calls, inputs of failed calls, reads a later change made stale) and gives the model a compress tool for spans it has finished with. Every change is an append-only context_edit entry, so the session file keeps the original, and a compressed span is archived for obs_recall.
Pruning rewrites history, and a rewrite invalidates the provider's prompt cache from the first changed message onward. The package therefore plans edits on every request but applies them in batches: when the cache is already cold, when the saving pays back the cache rewrite (see Choose), just before a compaction, or together with a compress call that breaks the cache anyway.
Index ¶
- Constants
- func Horizon(requests int) int
- func Mark(messages []agent.AgentMessage, ordinals []int) []agent.AgentMessage
- func Marker(ordinal int) string
- type CompressHost
- type CompressPlan
- type CompressTool
- func (t *CompressTool) Execute(_ context.Context, toolCallID string, params json.RawMessage, ...) (agent.AgentToolResult, error)
- func (t *CompressTool) ExecutionMode() agent.ToolExecutionMode
- func (t *CompressTool) Label() string
- func (t *CompressTool) Name() string
- func (t *CompressTool) Schema() ai.ToolSchema
- type Config
- type Edit
- type Item
- type Ordinals
- type Policy
Constants ¶
const ( StrategyDedupe = "dedupe" StrategyPurgeErrors = "purgeErrors" StrategySupersede = "supersedeReads" StrategyCompress = "compress" )
Strategy names, as reported in savings and edits.
const ( // MinEditTokens is the smallest saving worth an edit entry. MinEditTokens = 40 // PurgeAfterAssistants is how many later assistant messages show the // model has moved on from a failed call. PurgeAfterAssistants = 2 )
const CacheTTL = 5 * time.Minute
CacheTTL is how long a provider keeps a prompt prefix cached after its last use (Anthropic's default, and the low end of OpenAI's automatic caching).
const CompressToolName = "compress"
CompressToolName is the model-invoked span compressor.
const DefaultCompressThreshold = 0.4
DefaultCompressThreshold is the context-window fraction at which the compress tool is offered.
const LongCacheTTL = time.Hour
LongCacheTTL is the long prompt cache's lifetime (Anthropic's 1 hour; the low end of the long caches).
const LongCacheWriteReadRatio = 20
LongCacheWriteReadRatio is a long-cache write's cost in cache reads (Anthropic: 2x input to write, 0.1x to read).
const MinBatchTokens = 2000
MinBatchTokens is the smallest saving that justifies breaking a warm cache on its own.
const SettingsKey = "contextPruning"
SettingsKey is the settings.json key this package owns.
Variables ¶
This section is empty.
Functions ¶
func Horizon ¶
Horizon estimates the requests left in a run from the requests already in it (a session that has run long tends to run on), bounded to [4, 32].
func Mark ¶
func Mark(messages []agent.AgentMessage, ordinals []int) []agent.AgentMessage
Mark prefixes each user and tool-result message that has an ordinal with its marker. ordinals is parallel to messages; 0 means no marker. Messages are copied, never mutated. The marker depends only on the entry, so a marked message is byte-identical on every request.
Types ¶
type CompressHost ¶
type CompressHost interface {
// CompressItems returns the projected context the model sees.
CompressItems() []Item
// ArchiveSpan stores the original span for obs_recall and returns its
// id, or "" when there is no archive.
ArchiveSpan(key, text string) string
// QueueCompress records edits to apply before the next request. It
// refuses a span that overlaps one already queued.
QueueCompress(edits []Edit) error
}
CompressHost gives the compress tool the live context.
type CompressPlan ¶
CompressPlan is the edits for one compress call.
func PlanCompress ¶
func PlanCompress(items []Item, from, to int, summary, toolCallID string, archive func(key, text string) string) (CompressPlan, error)
PlanCompress resolves a span by marker numbers and returns the edits that replace it with summary. The span grows to whole tool-call batches, so a call is never separated from its result: a span that starts at a tool result starts at the assistant message that made the call, and a span that ends at a tool result ends after the batch's last result. The summary goes into the span's first editable message. When that is an assistant message, its tool calls stay (with short arguments) and their results become stubs, which keeps both message alternation and call/result pairing valid for every provider.
type CompressTool ¶
type CompressTool struct {
Host CompressHost
}
CompressTool lets the model replace a finished span with its summary.
func (*CompressTool) Execute ¶
func (t *CompressTool) Execute(_ context.Context, toolCallID string, params json.RawMessage, _ agent.ToolUpdateCallback) (agent.AgentToolResult, error)
func (*CompressTool) ExecutionMode ¶
func (t *CompressTool) ExecutionMode() agent.ToolExecutionMode
func (*CompressTool) Label ¶
func (t *CompressTool) Label() string
func (*CompressTool) Name ¶
func (t *CompressTool) Name() string
func (*CompressTool) Schema ¶
func (t *CompressTool) Schema() ai.ToolSchema
Schema is deliberately tiny: the tool is declared on every request once offered, so each byte is paid again and again. It lists no "required" array; Execute rejects a call that omits a field.
type Config ¶
type Config struct {
Enabled bool
Dedupe bool
PurgeErrors bool
SupersedeReads bool
Compress bool
// CompressThreshold is the fraction of the context window in use before
// the compress tool is offered.
CompressThreshold float64
}
Config selects the strategies. Every strategy defaults on.
func ParseConfig ¶
func ParseConfig(global, project json.RawMessage) (Config, []string)
ParseConfig layers the global then the project contextPruning section over the defaults. A malformed layer is reported and skipped.
func (Config) CompressOn ¶
CompressOn reports whether the compress tool may be offered.
type Edit ¶
type Edit struct {
// Index is the item the edit changes.
Index int
TargetID string
// Replacement is the new content (a JSON string or content array), or
// nil to omit the entry from context.
Replacement json.RawMessage
Strategy string
// Saved is the estimated number of context tokens the edit removes.
Saved int
}
Edit is one planned context edit.
func Choose ¶
Choose selects which edits to apply now. mandatory edits (a compress the model asked for) always apply, and they break the cache from their first index, so automatic edits at or after that index ride along for free. An earlier automatic edit applies only when the saving over the horizon pays for rewriting the cache from its position:
saved * horizon >= (suffix tokens - saved) * (writeReadRatio - 1)
type Item ¶
type Item struct {
// EntryID is the session entry a context edit can target, or "" for
// messages no edit can change (system messages, summaries, shell runs).
EntryID string
// Ordinal is the marker number the compress tool addresses, or 0.
Ordinal int
Message agent.AgentMessage
Tokens int
// Edited reports that an earlier context edit already changed the entry.
Edited bool
}
Item is one model-visible message of the projected context.
func BuildItems ¶
func BuildItems(projection codingagent.SessionProjection, path []codingagent.SessionEntry, ordinals *Ordinals) []Item
BuildItems pairs each projected message with its source entry. path is the branch from the root to the leaf, which carries the context edits.
type Ordinals ¶
type Ordinals struct {
// contains filtered or unexported fields
}
Ordinals numbers the user and tool-result message entries of a branch in path order. A number never changes once given, so a marker rendered into a message stays byte-identical across requests and compactions.
func (*Ordinals) Update ¶
func (o *Ordinals) Update(path []codingagent.SessionEntry) map[string]int
Update extends the numbering over path and returns it. A path that no longer extends the one seen before (a /tree move) is numbered afresh.
type Policy ¶
type Policy struct {
// Cold means the provider's prefix cache is already lost (idle past the
// TTL, just compacted, first request of a process), so every edit is free.
Cold bool
// Force applies every edit regardless of cost, as before a compaction
// the edits might avoid.
Force bool
// Horizon is the expected number of further requests that re-send the
// context.
Horizon int
// WriteReadRatio is the cost of a cache write in cache reads.
WriteReadRatio float64
}
Policy describes the cache state when a batch is considered.