Help us shape the future of Go! Take the 2026 Go Developer Survey

pruning

package
v0.2.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 5, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package pruning removes stale material from the model context without a model call (duplicate tool calls, inputs of failed calls, reads a later change made stale) and gives the model a compress tool for spans it has finished with. Every change is an append-only context_edit entry, so the session file keeps the original, and a compressed span is archived for obs_recall.

Pruning rewrites history, and a rewrite invalidates the provider's prompt cache from the first changed message onward. The package therefore plans edits on every request but applies them in batches: when the cache is already cold, when the saving pays back the cache rewrite (see Choose), just before a compaction, or together with a compress call that breaks the cache anyway.

Index

Constants

View Source
const (
	StrategyDedupe      = "dedupe"
	StrategyPurgeErrors = "purgeErrors"
	StrategySupersede   = "supersedeReads"
	StrategyCompress    = "compress"
)

Strategy names, as reported in savings and edits.

View Source
const (
	// MinEditTokens is the smallest saving worth an edit entry.
	MinEditTokens = 40
	// PurgeAfterAssistants is how many later assistant messages show the
	// model has moved on from a failed call.
	PurgeAfterAssistants = 2
)
View Source
const CacheTTL = 5 * time.Minute

CacheTTL is how long a provider keeps a prompt prefix cached after its last use (Anthropic's default, and the low end of OpenAI's automatic caching).

View Source
const CompressToolName = "compress"

CompressToolName is the model-invoked span compressor.

View Source
const DefaultCompressThreshold = 0.4

DefaultCompressThreshold is the context-window fraction at which the compress tool is offered.

View Source
const LongCacheTTL = time.Hour

LongCacheTTL is the long prompt cache's lifetime (Anthropic's 1 hour; the low end of the long caches).

View Source
const LongCacheWriteReadRatio = 20

LongCacheWriteReadRatio is a long-cache write's cost in cache reads (Anthropic: 2x input to write, 0.1x to read).

View Source
const MinBatchTokens = 2000

MinBatchTokens is the smallest saving that justifies breaking a warm cache on its own.

View Source
const SettingsKey = "contextPruning"

SettingsKey is the settings.json key this package owns.

Variables

This section is empty.

Functions

func Horizon

func Horizon(requests int) int

Horizon estimates the requests left in a run from the requests already in it (a session that has run long tends to run on), bounded to [4, 32].

func Mark

func Mark(messages []agent.AgentMessage, ordinals []int) []agent.AgentMessage

Mark prefixes each user and tool-result message that has an ordinal with its marker. ordinals is parallel to messages; 0 means no marker. Messages are copied, never mutated. The marker depends only on the entry, so a marked message is byte-identical on every request.

func Marker

func Marker(ordinal int) string

Marker is the id prefix the compress tool addresses.

Types

type CompressHost

type CompressHost interface {
	// CompressItems returns the projected context the model sees.
	CompressItems() []Item
	// ArchiveSpan stores the original span for obs_recall and returns its
	// id, or "" when there is no archive.
	ArchiveSpan(key, text string) string
	// QueueCompress records edits to apply before the next request. It
	// refuses a span that overlaps one already queued.
	QueueCompress(edits []Edit) error
}

CompressHost gives the compress tool the live context.

type CompressPlan

type CompressPlan struct {
	Edits     []Edit
	Saved     int
	ArchiveID string
}

CompressPlan is the edits for one compress call.

func PlanCompress

func PlanCompress(items []Item, from, to int, summary, toolCallID string, archive func(key, text string) string) (CompressPlan, error)

PlanCompress resolves a span by marker numbers and returns the edits that replace it with summary. The span grows to whole tool-call batches, so a call is never separated from its result: a span that starts at a tool result starts at the assistant message that made the call, and a span that ends at a tool result ends after the batch's last result. The summary goes into the span's first editable message. When that is an assistant message, its tool calls stay (with short arguments) and their results become stubs, which keeps both message alternation and call/result pairing valid for every provider.

type CompressTool

type CompressTool struct {
	Host CompressHost
}

CompressTool lets the model replace a finished span with its summary.

func (*CompressTool) Execute

func (*CompressTool) ExecutionMode

func (t *CompressTool) ExecutionMode() agent.ToolExecutionMode

func (*CompressTool) Label

func (t *CompressTool) Label() string

func (*CompressTool) Name

func (t *CompressTool) Name() string

func (*CompressTool) Schema

func (t *CompressTool) Schema() ai.ToolSchema

Schema is deliberately tiny: the tool is declared on every request once offered, so each byte is paid again and again. It lists no "required" array; Execute rejects a call that omits a field.

type Config

type Config struct {
	Enabled        bool
	Dedupe         bool
	PurgeErrors    bool
	SupersedeReads bool
	Compress       bool
	// CompressThreshold is the fraction of the context window in use before
	// the compress tool is offered.
	CompressThreshold float64
}

Config selects the strategies. Every strategy defaults on.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig turns every strategy on.

func ParseConfig

func ParseConfig(global, project json.RawMessage) (Config, []string)

ParseConfig layers the global then the project contextPruning section over the defaults. A malformed layer is reported and skipped.

func (Config) Automatic

func (c Config) Automatic() bool

Automatic reports whether any no-model strategy runs.

func (Config) CompressOn

func (c Config) CompressOn() bool

CompressOn reports whether the compress tool may be offered.

type Edit

type Edit struct {
	// Index is the item the edit changes.
	Index    int
	TargetID string
	// Replacement is the new content (a JSON string or content array), or
	// nil to omit the entry from context.
	Replacement json.RawMessage
	Strategy    string
	// Saved is the estimated number of context tokens the edit removes.
	Saved int
}

Edit is one planned context edit.

func Choose

func Choose(auto, mandatory []Edit, items []Item, p Policy) []Edit

Choose selects which edits to apply now. mandatory edits (a compress the model asked for) always apply, and they break the cache from their first index, so automatic edits at or after that index ride along for free. An earlier automatic edit applies only when the saving over the horizon pays for rewriting the cache from its position:

saved * horizon >= (suffix tokens - saved) * (writeReadRatio - 1)

func Plan

func Plan(cfg Config, items []Item, cwd string) []Edit

Plan returns the automatic edits for items: dedupe, errored-input purge and superseded reads, as cfg enables them. Items that already carry an edit are never changed again. cwd resolves relative paths.

type Item

type Item struct {
	// EntryID is the session entry a context edit can target, or "" for
	// messages no edit can change (system messages, summaries, shell runs).
	EntryID string
	// Ordinal is the marker number the compress tool addresses, or 0.
	Ordinal int
	Message agent.AgentMessage
	Tokens  int
	// Edited reports that an earlier context edit already changed the entry.
	Edited bool
}

Item is one model-visible message of the projected context.

func BuildItems

func BuildItems(projection codingagent.SessionProjection, path []codingagent.SessionEntry, ordinals *Ordinals) []Item

BuildItems pairs each projected message with its source entry. path is the branch from the root to the leaf, which carries the context edits.

type Ordinals

type Ordinals struct {
	// contains filtered or unexported fields
}

Ordinals numbers the user and tool-result message entries of a branch in path order. A number never changes once given, so a marker rendered into a message stays byte-identical across requests and compactions.

func (*Ordinals) Update

func (o *Ordinals) Update(path []codingagent.SessionEntry) map[string]int

Update extends the numbering over path and returns it. A path that no longer extends the one seen before (a /tree move) is numbered afresh.

type Policy

type Policy struct {
	// Cold means the provider's prefix cache is already lost (idle past the
	// TTL, just compacted, first request of a process), so every edit is free.
	Cold bool
	// Force applies every edit regardless of cost, as before a compaction
	// the edits might avoid.
	Force bool
	// Horizon is the expected number of further requests that re-send the
	// context.
	Horizon int
	// WriteReadRatio is the cost of a cache write in cache reads.
	WriteReadRatio float64
}

Policy describes the cache state when a batch is considered.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL