Documentation
¶
Overview ¶
Command search-bench scores the full-text search ranking (index.SearchChunks, behind both the search CLI and the search_docs MCP tool) against labeled queries over frozen crawls of real documentation sites:
go run ./tools/search-bench go run ./tools/search-bench -out .audit/search-bench/runs/<date>-<label>.json go run ./tools/search-bench -baseline .audit/search-bench/runs/<date>-baseline.json
Corpora are not committed. Get them once, from the repo root, with
doc-scraper crawl -config tools/search-bench/crawl.yaml -all-sites
which leaves .audit/search-bench/corpora/<site_key>/pages.jsonl. Each run indexes every corpus into its own temp index, so bm25 statistics come from that corpus alone. A recrawl changes the corpus: the run then fails if an expected URL is gone, and expected pages need re-checking either way.
Queries live in tools/search-bench/queries/<site_key>.json. Relevance is judged per page, in the order SearchChunks returns: chunks collapse to their page's best rank. Scores never change the exit code.
Source Files