Documentation
¶
Overview ¶
Package search wraps the embedded Bleve index used for fast filename (and optionally content) lookup across all storages.
The index lives at {data_dir}/search.bleve. It is opened lazily on first IndexNode/Search call. If Bleve cannot open or create the index, Search degrades to nil (callers should fall back to SQL LIKE).
Package search — name.go ¶
Forgiving filename matching: separator-blind normalisation, multi-word wildcards, typo tolerance, an explicit ranking contract, and the `tag:` query filter.
Motivated by issue #15, which reported that `main go` did not find `main.go`. Measuring it turned up something narrower and stranger than the report: single words always worked, and `main go` / `foo bar` already worked, while `invoice 2026` did not find `invoice_2026.pdf`. The reason is tokenisation, not the number of words. Bleve's standard analyzer splits `invoice_2026.pdf` into [invoice_2026, pdf] — an underscore JOINS — and `foo-bar.txt` into [foo, bar.txt] — a hyphen SPLITS. So which separator the file happened to use decided whether the search worked, a distinction no user can predict.
The fix is to stop letting the analyzer decide: every name and every query is normalised to the same separator-blind form before it reaches the index, so `.`, `-`, `_` and a space are all the same character.
Package search — rebuild.go ¶
The self-repairing index.
v0.29.0 (issue #15) added two indexed fields, name_norm and path_norm, and shipped without a rebuild: an index written by an older build was opened as is, reported `needs_rebuild: true` in the admin stats, and was otherwise left alone. The reasoning was sound as far as it went — a rebuild started from an EMPTY index, and extracted file content lives only in the index (the database holds no copy), so rebuilding on an operator's behalf would have traded a recall improvement nobody asked for against a content-search blackout nobody was warned about.
The consequence was worse than the cure: the default experience of the upgrade, on every existing installation, was "nothing changed". The reporter of issue #15 came back and said exactly that, and was right — he was measuring an un-rebuilt index.
So the blackout is removed instead of the rebuild. Two things make an automatic rebuild safe here:
The replacement index is built ALONGSIDE the live one and swapped in atomically. The old index answers every query until the new one is complete and verified; the swap itself happens under the index write lock, so a concurrent search waits a few milliseconds rather than seeing an empty or closed index. Nothing observes a half-built index.
Extracted text is CARRIED OVER document by document from the live index into the replacement, instead of being dropped and re-derived. The rebuild changes how names are indexed; it has no reason to touch content at all. Files whose bytes actually drifted are re-enqueued for extraction the way any other stale file is.
Package search — scorer.go ¶
Subsequence scoring for filenames, ported from VS Code's Quick Open (src/vs/base/common/fuzzyScorer.ts, MIT).
Why a second matching layer exists at all. v0.29.0 answered issue #15 with separator-blind normalisation plus substring matching, and the reporter came back with three things it still could not do:
"Fuzzy is distance matching. You shipped string matching. [...] Code/main.go and example/main.go are the same thing to the search, and `Code main` returns nothing. [...] with real fuzzy the word order shouldn't matter either, `main code` should still find Code/main.go"
All three are the same missing piece: nothing in the pipeline scored a candidate. Bleve decided IF a document matched, and a merged Bleve score decided the order — and a Bleve score knows nothing about where in the filename the match landed, whether the folder or the file answered, or whether every word the user typed was answered at all.
So the shape is: Bleve stays the candidate generator (cheap recall from the normalised fields, the wildcards and the typo pass), and this file re-ranks what comes back. It is also the FILTER — a candidate where some query piece matches nothing is dropped, which is what turns `Code main` from nine results into one.
⚠ What this file can NOT do, and it is worth being plain about it: it only ever sees candidates Bleve produced. Subsequence SCORING is not subsequence RECALL. `mgo` does not find `main.go`, because no Bleve query retrieves it — see docs/SEARCH.md, "What fuzzy does not mean here", for why widening recall needs an index change rather than a scoring change.
Index ¶
- Constants
- Variables
- func ContentFingerprint(n *model.Node) string
- func MatchName(q, name string) bool
- func NormWords(s string) []string
- func Normalize(s string) string
- type Fallback
- type Filter
- type Hit
- type Index
- func (i *Index) AutoRebuildIfStale(store NodeLister, enabled bool) bool
- func (i *Index) Close() error
- func (i *Index) DeleteNode(_ context.Context, id int64) error
- func (i *Index) Enabled() bool
- func (i *Index) IndexNode(ctx context.Context, n *model.Node) error
- func (i *Index) IndexNodeContent(_ context.Context, n *model.Node, content string) error
- func (i *Index) IndexNodes(ctx context.Context, nodes []*model.Node) error
- func (i *Index) NeedsRebuild() bool
- func (i *Index) Rebuild(ctx context.Context, store NodeLister, opts RebuildOptions) error
- func (i *Index) RebuildAll(ctx context.Context, store NodeLister) error
- func (i *Index) RebuildAllWithContent(ctx context.Context, store NodeLister) error
- func (i *Index) Rebuilding() bool
- func (i *Index) SafeSearch(ctx context.Context, query string, limit int) []Hit
- func (i *Index) SafeSearchFiltered(ctx context.Context, query string, limit int, scope Scope, f *Filter) []Hit
- func (i *Index) SafeSearchScoped(ctx context.Context, query string, limit int, scope Scope) []Hit
- func (i *Index) Search(ctx context.Context, query string, limit int) ([]Hit, error)
- func (i *Index) SearchFiltered(ctx context.Context, q string, limit int, scope Scope, f *Filter) ([]Hit, error)
- func (i *Index) SearchPage(_ context.Context, q string, limit int, scope Scope, f *Filter) ([]Hit, bool, error)
- func (i *Index) SearchScoped(ctx context.Context, query string, limit int, scope Scope) ([]Hit, error)
- func (i *Index) SetContentHook(h func(ctx context.Context, n *model.Node))
- func (i *Index) StartRebuild(store NodeLister, opts RebuildOptions) error
- func (i *Index) Stats() IndexStats
- type IndexStats
- type NameMatcher
- type NameScore
- type NodeLister
- type NodeSearcher
- type Parsed
- type PreparedQuery
- type RebuildOptions
- type Scope
- type Tier
Constants ¶
const ( MatchedName = "name" MatchedContent = "content" MatchedBoth = "both" )
Values for Hit.Matched (frozen v0.2 API contract).
const ( // PathIdentityScore is awarded when the query IS the path. PathIdentityScore = 1 << 18 // LabelPrefixScoreThreshold is the baseline for a piece that matches // the start of the filename. LabelPrefixScoreThreshold = 1 << 17 // LabelScoreThreshold is the baseline for a piece that matches the // filename anywhere else. LabelScoreThreshold = 1 << 16 )
Score thresholds, verbatim from VS Code. They are what keeps the classes apart no matter how the per-character bonuses add up: a label match always outranks a folder-path match, and a label PREFIX match always outranks a label match somewhere in the middle, because the baseline they start from differs by a factor of two.
The issue asked for exact matches to keep ranking first; these thresholds plus Tier are how that stays true rather than emerging from bonus arithmetic.
const FallbackOverFetch = 4
FallbackOverFetch is how many times `limit` rows a caller should ask the database for, since Accepts drops some of them afterwards.
Variables ¶
var ErrIndexDisabled = errors.New("search: index disabled")
ErrIndexDisabled is returned when there is no live index to rebuild.
var ErrNoDiskSpace = errors.New("search: not enough free disk space to rebuild the index")
ErrNoDiskSpace is returned when the filesystem holding the index cannot hold a second copy of it. Failing loudly and continuing to serve the old index is the right answer here; filling the disk is not.
var ErrRebuildInProgress = errors.New("search: rebuild already in progress")
ErrRebuildInProgress is returned when a rebuild is already running. Manual (POST /api/admin/search/rebuild) and automatic rebuilds share one guard, held on the Index rather than on the HTTP handler — the handler's own atomic could only ever see the rebuilds the handler started.
Functions ¶
func ContentFingerprint ¶
ContentFingerprint identifies a node's content version — used to decide whether the indexed content is stale. It is model.Node.ContentFingerprint, the rule the thumbnail pipeline uses too.
func NormWords ¶
NormWords is Normalize split into its words — the unit every separator-blind sub-query is built per.
func Normalize ¶
Normalize maps a filename — or a raw user query — onto the separator-blind form both sides of a search are compared in: lower-cased, with every run of non-alphanumeric characters collapsed to a single space.
invoice_2026.pdf -> "invoice 2026 pdf" foo-bar.txt -> "foo bar txt" annual report 2025.docx -> "annual report 2025 docx"
This is deliberately aggressive: the whole point is that a user typing `invoice 2026` must not have to know which of `.`, `-`, `_` or a space the file actually used. The ORIGINAL name stays indexed in the `name` field, so a query that leans on punctuation the normaliser removes (`c++`) still has the legacy wildcard path to land on.
Letters and digits are kept by Unicode class, not by ASCII range, so `rapor-şubat.txt` normalises to `rapor şubat txt` instead of losing its Turkish characters. They are folded by internal/namefold — the one rule for when two spellings are the same text, shared with the fallback's database query, the scorer and tags — so `IŞIK`, `Işık` and `ışık` are one word (`işik`), which strings.ToLower would have made two.
The input is composed first (namefold.Canonical), and a combining mark is kept as part of its word. Before, a decomposed `Gürel` — `Gu`, U+0308, `rel` — normalised to the two words `gu rel`, because a mark is neither a letter nor a digit. Composition removes most marks; the ones that survive it have no precomposed form (every Devanagari vowel sign is one) and belong to the letter before them just the same.
Types ¶
type Fallback ¶
type Fallback struct {
// Words is every normalised query word, each once; a row must contain
// them all. Empty for a query with no letter or digit in it (`***`).
Words []string
// contains filtered or unexported fields
}
Fallback is how a query is answered WITHOUT the Bleve index.
The index-less install is a different code path, not a different product: an operator running with FILEX_SEARCH_ENABLED=false should see `invoice 2026` find `invoice_2026.pdf` exactly like everybody else. A LIKE cannot do that on its own — `%invoice 2026%` matches no filename that used a different separator, which is the whole bug from issue #15 — so the separator-blind part happens in two steps:
- Candidates asks the database for the rows whose NAME holds EVERY word of the query (Store.SearchNodes), compared the way internal/namefold compares — composed, the four i's one letter, case folded — on the stored side as on this one, and ranked before the LIMIT.
- Accepts re-checks the FULL query against each returned row with the same scorer the index path uses, which also ranks it.
⚠ Step 1 used to send only the longest word and leave the others to step 2. The database then returned the first rows by name that held that one word — at most a thousand of them — and a row answering the whole query but sorting after them was never looked at. Measured on a production catalogue on 2026-09-24 (GitHub PR #46): the longest word of a typed filename was the prefix every file in the storage carried, it was in 64 483 of 169 471 names, and the two files being looked for sat at rows 4 128 and 33 623. So the more of a name somebody typed, the less they found; the file's own name, pasted byte for byte, found nothing. With every word in the query, the LIMIT counts rows that can actually answer it.
⚠ Every word is matched against the NAME, never the path. The name is in an index (storage_id, parent_id, name), so SQLite rejects a row without reading it; a condition on the path reads every candidate row from the table. Measured on the same catalogue: an eleven-word query took 1.06 s with the words matched against the path and 0.12 s against the name. A word that appears only in a folder name is therefore not found without the index — the anchor was matched against the name before too.
What the fallback deliberately does NOT get is typo tolerance: edit distance is not something a LIKE can express, and faking it with more patterns would turn one scan into many. Said out loud in docs/SEARCH.md rather than left for somebody to discover.
func PlanFallback ¶
PlanFallback builds the two-step plan described on Fallback.
The database is asked for three things (model.NameMatch):
- Words: every word, folded (namefold), longest first — a row is turned away by the first word it lacks, and a long word is the likelier one to be rare.
- Runs: for each word that is not all plain letters (namefold.AllPlain — those the store looks for natively as they are), the longest run of letters and digits that every stored spelling of it holds as it is (sharedRun: `arch` for `archive`, `rel` for `gürel`). The engine checks those with its own LIKE on the stored bytes before the costlier comparison through the normaliser: on SQLite that comparison is a Go function, a few microseconds a row, and the run turns most rows away before it is called.
- Prefer: the longest word, to rank the rows before the LIMIT — names that are the word, then names starting with it.
A query with no letter or digit in it (`***`, `---`) asks for the query itself, trimmed, as one piece of text.
func (Fallback) Accepts ¶
Accepts reports whether a row the database returned really satisfies the whole query — the subsequence scorer's verdict, so the filename and the folders above it are weighed exactly as they are on the index path and a row every piece cannot answer is dropped.
func (Fallback) Candidates ¶
func (f Fallback) Candidates(ctx context.Context, store NodeSearcher, storageID int64, limit int) ([]*model.Node, error)
Candidates fetches the rows of one storage that step 2 has to look at. Every caller goes through here — the toolbar search, /api/files/search and the AI/MCP name search — so no surface can go back to asking the database for one word.
type Filter ¶
type Filter struct {
// Restrict limits results to IncludeIDs. Restrict with an empty
// IncludeIDs means "the filter matched nothing", and the search
// honours that: a tag that does not exist returns no results rather
// than being quietly ignored and returning everything.
Restrict bool
IncludeIDs []int64
ExcludeIDs []int64
// Accept, when set, is asked about every candidate BEFORE it is counted
// against `limit` (filex 0.54, audit D6): a search's narrowing (kind,
// size, date, folder, owner - internal/nodefilter) is a property of the
// catalogue row, which the index does not hold, so the caller answers it.
// A candidate it refuses is not a result and takes no place in the page.
// The candidate window grows to searchMaxFilteredFetch while it is set.
Accept func(nodeID int64) bool
// AcceptNode is the same narrowing for a caller that already holds the
// row (the index-less fallback, a bare `tag:` listing, the MCP name
// search): no second lookup. Set together with Accept.
AcceptNode func(n *model.Node) bool
}
Filter narrows a search to a set of node IDs — how the `tag:` filter reaches the index without tags having to be indexed.
Tags live in the node_meta table and change without the node itself being re-indexed, so a tag copied into the Bleve document would go stale the moment somebody re-tagged a file. The caller resolves the tag to node IDs against the database (always current) and hands the set down here, where it becomes a conjunction — so the filter runs INSIDE the engine and `limit` still counts filtered results.
type Hit ¶
type Hit struct {
NodeID int64
Score float64
// Snippet is a short plain-text fragment around a content match, with
// the matched terms wrapped in « » (empty for name-only hits, no HTML).
Snippet string
// Matched reports which side(s) hit: "name" | "content" | "both".
Matched string
// Tier is the rank bucket this hit was sorted into (issue #15). It is
// deliberately NOT on the wire — the HTTP response shape is frozen —
// but it is what makes "exact matches rank first" an assertable
// contract instead of a hope about merged Bleve scores.
Tier Tier
}
Hit is a single search result.
type Index ¶
type Index struct {
// FreeBytes probes free disk space on the filesystem holding the
// index. Swapped in tests to exercise the guard without filling a
// disk (same shape as staging.Area.FreeBytes).
FreeBytes func(dir string) (uint64, error)
// contains filtered or unexported fields
}
Index is the search facade.
func Open ¶
Open returns an Index ready to use. Pass empty path to disable.
Upgrade behaviour (issue #15 added two indexed fields): an index written by an older build is opened AS IS and keeps serving. The document schema stamped inside it is compared with the one this build writes, and a mismatch is recorded, logged, and reported through the admin stats endpoint — but the repair itself is started later, from the server bootstrap (Index.AutoRebuildIfStale), because it needs the node store and the content queue and neither exists at this point.
Until the replacement is live, search returns everything it returned before the upgrade: the pre-#15 sub-queries are still part of every query, so the change adds recall and never removes any.
⚠ v0.29.0 stopped here, and that was the bug. It shipped better filename matching that could not reach a single document any existing installation already had, said so only in an admin endpoint, and the reporter of issue #15 upgraded and correctly reported that nothing had changed. What made an automatic rebuild unacceptable then — a rebuild started from an empty index and extracted content lives only there — is fixed in rebuild.go, not worked around.
func (*Index) AutoRebuildIfStale ¶
func (i *Index) AutoRebuildIfStale(store NodeLister, enabled bool) bool
AutoRebuildIfStale starts a background self-repair when the index on disk was written by an older document schema. Reports whether it started.
enabled=false is FILEX_SEARCH_AUTO_REBUILD=0 and is honoured before any work starts — no disk touched, no replacement index created.
func (*Index) DeleteNode ¶
DeleteNode removes a node from the index.
func (*Index) Enabled ¶
Enabled reports whether a live Bleve index is wired (false = the server runs in SQL LIKE fallback mode).
func (*Index) IndexNode ¶
IndexNode adds or updates a node entry. Previously extracted content is carried over (a rename/move must not wipe it); when the content fingerprint drifted, the content hook fires so extraction is re-queued.
func (*Index) IndexNodeContent ¶
IndexNodeContent updates the node's document with extracted content. The metadata fields are re-supplied from n (the authoritative row) so the content update never clobbers them — Bleve has no partial update, the whole doc is replaced. Metadata indexing stays synchronous elsewhere; this lands later, from the content_index queue job.
func (*Index) IndexNodes ¶ added in v0.45.0
IndexNodes is IndexNode for many nodes in ONE index write (issue #70).
⚠ Why it exists: every Bleve write waits until its segment is on disk, so the catalogue indexing a folder of a hundred new files one IndexNode at a time waited for a hundred of them — measured on Windows, about 10 ms a file, the largest single cost of cataloguing a tree. The sync walk and the lazy catalogue hand a whole folder's rows over here instead. Each document is built exactly as IndexNode builds it, and the content hook fires per node on the same rule.
func (*Index) NeedsRebuild ¶
NeedsRebuild reports whether the on-disk index predates the current document schema. See Open.
func (*Index) Rebuild ¶
func (i *Index) Rebuild(ctx context.Context, store NodeLister, opts RebuildOptions) error
Rebuild runs a rebuild synchronously (tests, and any caller that wants to wait). Returns ErrRebuildInProgress when one is already running.
func (*Index) RebuildAll ¶
func (i *Index) RebuildAll(ctx context.Context, store NodeLister) error
RebuildAll reindexes every node row into a replacement index and swaps it in. The Store interface is referenced via an opaque interface to avoid an import cycle. Text already extracted is carried over from the live index; nothing is re-extracted (use RebuildAllWithContent for that). See rebuild.go for the swap.
func (*Index) RebuildAllWithContent ¶
func (i *Index) RebuildAllWithContent(ctx context.Context, store NodeLister) error
RebuildAllWithContent is RebuildAll plus content re-extraction: every eligible file is re-enqueued for extraction once the replacement index is live.
func (*Index) Rebuilding ¶
Rebuilding reports whether a rebuild is currently running. Surfaced by the admin stats endpoint so the UI can say "rebuilding" instead of looking broken.
func (*Index) SafeSearch ¶
SafeSearch wraps Search and logs errors instead of bubbling them. Useful in handlers that want a "best effort" search.
func (*Index) SafeSearchFiltered ¶
func (i *Index) SafeSearchFiltered(ctx context.Context, query string, limit int, scope Scope, f *Filter) []Hit
SafeSearchFiltered is SearchFiltered with errors logged instead of returned.
func (*Index) SafeSearchScoped ¶
SafeSearchScoped is SearchScoped with errors logged instead of returned.
func (*Index) Search ¶
Search returns top-N name/path matches for the query string — the legacy, name-scoped entry point (see SearchScoped for content search).
Falls back to nil result + nil error when index is disabled — callers should treat that as "no search engine, do a SQL LIKE instead".
See nameQuery for how a query is built and Tier for how its results are ordered.
func (*Index) SearchFiltered ¶
func (i *Index) SearchFiltered(ctx context.Context, q string, limit int, scope Scope, f *Filter) ([]Hit, error)
SearchFiltered is SearchScoped with an optional node-ID Filter (how the `tag:` filter is applied — see Filter).
Ordering is the Tier contract, best tier first, Bleve score descending within a tier, node ID as the final tiebreak so the same query always comes back in the same order. Because TierContent sorts last, the pre-v0.2 frozen contract — name hits before content-only hits — is preserved by construction rather than by luck: a document that matched on both keeps its NAME tier and stays ahead of content-only hits, and is still reported once, with Matched="both" and its snippet.
Two passes run against the name fields. The strict pass is the legacy disjunction plus the separator-blind additions; the typo-tolerant pass runs ONLY when the strict pass came back short of `limit`. That is both the cheap and the quiet choice: fuzziness is the expensive part of the query, and always-on fuzziness would pad a perfectly good result list with near-misses nobody asked for.
func (*Index) SearchPage ¶ added in v0.54.0
func (i *Index) SearchPage(_ context.Context, q string, limit int, scope Scope, f *Filter) ([]Hit, bool, error)
SearchPage is SearchFiltered that also says whether more may match than came back: the page was cut at `limit`, or a candidate window Bleve filled held more matches than it returned (so a Filter.Accept that turned rows away may have left matches outside the window unseen).
func (*Index) SearchScoped ¶
func (i *Index) SearchScoped(ctx context.Context, query string, limit int, scope Scope) ([]Hit, error)
SearchScoped runs the query against the requested scope, unfiltered.
func (*Index) SetContentHook ¶
SetContentHook wires the callback fired for file nodes whose content needs (re)extraction — see Index.contentHook. Call before serving.
func (*Index) StartRebuild ¶
func (i *Index) StartRebuild(store NodeLister, opts RebuildOptions) error
StartRebuild runs a rebuild in the background and returns as soon as it has started (or refuses, with ErrRebuildInProgress).
The goroutine gets context.Background() on purpose. The manual rebuild is triggered from an HTTP handler and chi cancels r.Context() the moment the handler returns, which would kill the reindex before it processed a single row; the same trap applies to the boot context the automatic path is started from, which is cancelled on shutdown paths a rebuild has no reason to care about.
func (*Index) Stats ¶
func (i *Index) Stats() IndexStats
Stats returns DocCount + on-disk size for the index.
type IndexStats ¶
type IndexStats struct {
DocCount uint64
// Files and Folders split DocCount by node type. ⚠ DocCount counts every
// node — folders too — while the Panel's "indexed files" counts files, so
// the two pages disagreed about the same index (28 on the Panel, 37 on
// Search: 28 files and 9 folders; release-candidate sweep, 2026-09-21).
// The Search page reports these two instead of the bare document count.
Files uint64
Folders uint64
SizeBytes int64
// LastUpdated is when the index was built (RFC 3339), or "" for an
// index built before the stamp existed.
LastUpdated string
// NeedsRebuild is true when the index was written by a build with an
// older document schema. Search still works — see Open — but the
// documents already in it cannot answer the separator-blind or
// typo-tolerant half of a query until they are reindexed. It stays
// true for the whole of a rebuild: until the swap, the index
// answering queries really is the old one.
NeedsRebuild bool
// Rebuilding is true while a replacement index is being built, by
// either the admin endpoint or the automatic schema repair.
Rebuilding bool
// LastRebuildAt is when the latest rebuild of this process ended, nil
// when none has; LastRebuildError is why it did not go live ("" when it
// did). Rebuilding turns false either way, so it alone cannot say which.
LastRebuildAt *time.Time
LastRebuildError string
}
IndexStats summarizes the Bleve index size + last update.
type NameMatcher ¶ added in v0.54.0
type NameMatcher struct {
// contains filtered or unexported fields
}
NameMatcher is the search's own rule for "does this NAME answer what was typed", for a name on its own: no index, no folder above it.
⚠ Why it exists (filex 0.54, audit D5). The explorer's "Filter in this folder" box used to narrow the rows in hand with its own rule, a substring over an accent-stripped copy of the name, and its comment said that was the search's rule. It was not: the search keeps accents ("müşteri" is not "musteri", internal/namefold), treats `.`, `-`, `_` and a space as one separator ("invoice 2026" finds "invoice_2026.pdf") and wants every word. So the box and the search disagreed on the same folder. The box now asks the server (POST /api/files/search/match), and the server answers with this.
The rule is the index-less search's (Fallback): every normalised word of the query inside the folded name, then the subsequence scorer's verdict over the name alone. A query with no letter or digit (`***`) asks for itself as text.
func NewNameMatcher ¶ added in v0.54.0
func NewNameMatcher(q string) NameMatcher
NewNameMatcher prepares q once, for many names.
func (NameMatcher) Empty ¶ added in v0.54.0
func (m NameMatcher) Empty() bool
Empty reports whether the query narrows nothing (blank).
func (NameMatcher) Match ¶ added in v0.54.0
func (m NameMatcher) Match(name string) bool
Match reports whether name answers the query.
type NameScore ¶
type NameScore struct {
// OK is false when some query piece matched nothing. The candidate is
// not a result — this is the filter half of the scorer.
OK bool
// Score orders candidates WITHIN a tier. Bigger is better.
Score int
// Tier is the rank bucket, and it is what orders candidates ACROSS
// classes. See Tier.
Tier Tier
}
NameScore is one candidate's verdict.
type NodeLister ¶
NodeLister is the slim Store contract RebuildAll needs.
type NodeSearcher ¶
type NodeSearcher interface {
SearchNodes(ctx context.Context, storageID int64, m model.NameMatch, limit int) ([]*model.Node, error)
}
NodeSearcher is the part of the store a fallback runs against. It is an interface for the same reason NodeLister is: this package cannot import the store.
type Parsed ¶
type Parsed struct {
// Text is the query with every tag token removed. It may be empty (a
// pure `tag:x` query), in which case the caller should list the
// tagged nodes rather than run a text search.
Text string
// Tags must ALL be present on a node for it to survive the filter.
Tags []string
// ExcludeTags (`-tag:x`) drop a node when ANY of them is present.
ExcludeTags []string
}
Parsed is a raw query string split into free text and tag filters.
Tags are a FILTER, not a search term: `main go tag:source` means "the name match `main go`, restricted to nodes tagged source", never "also look for the word source". Filters narrow, so several tags are ANDed.
func ParseQuery ¶
ParseQuery splits `foo bar tag:source -tag:archive` into its text and tag parts.
Rules, all deliberate and mirrored in docs/SEARCH.md:
- the `tag:` prefix is case-insensitive, and so is the tag VALUE. The tags endpoint lower-cases on write, so a query that did not lower-case would silently match nothing;
- `tag:"two words"` is supported, because the tags endpoint accepts any string up to 64 characters and people do use spaces;
- a bare `tag:` with no value is NOT a filter. It stays in the text, so somebody looking for a file actually called `tag:` still can.
func (Parsed) HasTagFilter ¶
HasTagFilter reports whether any tag token was parsed out.
type PreparedQuery ¶
type PreparedQuery struct {
// contains filtered or unexported fields
}
PreparedQuery is a user query split into the pieces every candidate has to answer.
The split is on whitespace and it is the whole reason `main code` finds Code/main.go: each piece is matched independently, so the ORDER the user typed them in carries no meaning. That is VS Code's behaviour, and it is what the reporter asked for in as many words.
func PrepareQuery ¶
func PrepareQuery(raw string) PreparedQuery
PrepareQuery splits and normalises a raw query string.
⚠ The caller must have run ParseQuery first: `tag:` is a filter, and a `tag:source` token reaching this function would become a piece that no filename answers, so every candidate would be dropped and a perfectly good tag search would return nothing. Asserted by TestPrepareQuery_RawTagTokenWouldDropEverything.
func (PreparedQuery) Empty ¶
func (q PreparedQuery) Empty() bool
Empty reports whether the query has nothing to score with.
func (PreparedQuery) Rank ¶
func (q PreparedQuery) Rank(name, path string) Tier
Rank is RankName's body once the query has been prepared, so a caller ranking a whole result set prepares the query once instead of once per comparison.
func (PreparedQuery) ScoreName ¶
func (q PreparedQuery) ScoreName(name, path string) NameScore
ScoreName scores one candidate's filename and path against the query.
The label (the filename) and the description (the folders above it) are scored SEPARATELY, and the label is weighted an order of magnitude higher — that is VS Code's doScoreItemFuzzy, and it is the answer to "Code/main.go and example/main.go are the same thing to the search". The combined `folder/name` string is only scored for pieces the label alone could not answer.
The tier, unlike the score, is OURS. VS Code has no notion of an exact match distinct from a very good fuzzy one; issue #15 explicitly asked that exact filenames keep ranking first, so exact and prefix are still decided by the separator-blind rules RankName has used since v0.29.0, and the subsequence score orders what is left.
type RebuildOptions ¶
type RebuildOptions struct {
// ReExtract re-enqueues content extraction for EVERY eligible file.
// This is the manual `?content=1` contract: an operator asking for it
// has usually just added an extractor or changed the size cap, and
// wants the text derived again rather than copied.
ReExtract bool
// ExtractMissing re-enqueues extraction only for files whose content
// fingerprint drifted from what the index holds — what the automatic
// schema repair uses, because it already carried the text over.
ExtractMissing bool
// Reason names the caller in the logs ("admin", "schema-upgrade").
Reason string
}
RebuildOptions selects what a rebuild does about file content. The metadata half is not optional: every row is rewritten with the current document schema, which is the point of the exercise.
type Scope ¶
type Scope string
Scope selects which fields a search consults.
const ( ScopeAll Scope = "all" // names + content, name hits ranked first ScopeName Scope = "name" // filenames/paths only (legacy behavior) ScopeContent Scope = "content" // extracted file content only )
Search scopes (the `scope` query param of the search endpoints).
func ParseScope ¶
ParseScope maps a request string onto a Scope (default: all).
type Tier ¶
type Tier int
Tier is the rank bucket a hit falls in.
The issue asked that exact filename matches keep ranking first. Once fuzziness is in the mix that stops being something merged Bleve scores can be trusted to deliver — a two-edit fuzzy hit on a short filename can out-score an exact hit on a long one — so the order is decided HERE, in Go, and asserted by a test, rather than left to emerge from boost arithmetic.
Lower is better. TierContent is last, which is also how the pre-v0.2 "name hits before content-only hits" contract survives unchanged.
const ( TierExact Tier = iota // filename equals the query (ignoring case + separators) TierPrefix // filename starts with the query TierName // every query word appears in the filename TierPath // every query word appears in the path (folder hit) TierFuzzy // only the typo-tolerant pass produced this TierContent // matched inside the file, not in its name )
Ranking tiers, best first.
func RankName ¶
RankName classifies one name hit against the query. Both sides are compared in normalised form, so `invoice 2026` ranks as an exact match for a file called `invoice_2026.pdf` exactly as it would for one literally called `invoice 2026`.
Everything BELOW exact and prefix is decided by the subsequence scorer (scorer.go), so a hit's tier means the same thing whichever surface produced it. A candidate the scorer rejects outright lands in TierFuzzy here rather than disappearing, because RankName has to return a tier for every input; the index path calls ScoreName directly, which can also say "not a result at all".
Source Files
Directories