Documentation
¶
Overview ¶
Package vecstore provides a vector approximate nearest-neighbor (ANN) search interface and implementations.
The Index interface defines the contract for vector storage and search. Implementations include an in-memory brute-force index for testing (NewMemory) and HNSW for persisted ANN search.
This package follows the same pattern as [kv]: a generic interface with pluggable backends. For all-in-one deployment, use the built-in implementations. For distributed deployment, swap in a client that talks to Milvus, Qdrant, or similar.
Index ¶
- func CosineDistance(a, b []float32) float32
- type HNSW
- func (h *HNSW) BatchInsert(ids []string, vectors [][]float32) error
- func (h *HNSW) Close() error
- func (h *HNSW) Config() HNSWConfig
- func (h *HNSW) Delete(id string) error
- func (h *HNSW) Flush() error
- func (h *HNSW) Insert(id string, vector []float32) error
- func (h *HNSW) Len() int
- func (h *HNSW) Name() string
- func (h *HNSW) Remove() error
- func (h *HNSW) Save(w io.Writer) error
- func (h *HNSW) Search(query []float32, topK int) ([]Match, error)
- func (h *HNSW) SetEfSearch(ef int) error
- type HNSWConfig
- type HNSWLoadOptions
- type Index
- type Match
- type Memory
- func (m *Memory) BatchInsert(ids []string, vectors [][]float32) error
- func (m *Memory) Close() error
- func (m *Memory) Delete(id string) error
- func (m *Memory) Flush() error
- func (m *Memory) Insert(id string, vector []float32) error
- func (m *Memory) Len() int
- func (m *Memory) Search(query []float32, topK int) ([]Match, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CosineDistance ¶
CosineDistance computes the cosine distance between two vectors. Returns a value in [0, 2] where 0 means identical direction and 2 means opposite direction. Returns 2 if either vector has zero norm.
Types ¶
type HNSW ¶
type HNSW struct {
// contains filtered or unexported fields
}
HNSW is a Hierarchical Navigable Small World index implementing Index.
It provides approximate nearest-neighbor search with O(log n) query time by organizing vectors into a multi-layer navigable graph. Higher layers contain exponentially fewer nodes and act as "express lanes" for fast traversal; layer 0 contains all nodes for precise local search.
All methods are safe for concurrent use.
func LoadHNSW ¶
LoadHNSW deserializes an HNSW index from r. The returned index is ready for immediate use (Insert, Search, Delete).
func LoadHNSWWithOptions ¶
func LoadHNSWWithOptions(r io.Reader, opts HNSWLoadOptions) (*HNSW, error)
LoadHNSWWithOptions deserializes an HNSW index from r with caller-provided defensive limits. The returned index is ready for immediate use.
func NewHNSW ¶
func NewHNSW(cfg HNSWConfig) *HNSW
NewHNSW creates an empty HNSW index with the given configuration. Panics if cfg.Dim is not positive.
func OpenHNSW ¶
func OpenHNSW(objects objectstore.ObjectStore, name string, cfg HNSWConfig) (*HNSW, error)
OpenHNSW opens or creates a persistent HNSW index. The index is stored in objects under the given name. If the object exists, it is loaded and the dimension is validated against cfg. If the object does not exist, a new empty index is created.
Subsequent mutations mark the index as dirty, and HNSW.Flush / HNSW.Close write the index through objects.
func (*HNSW) BatchInsert ¶
BatchInsert adds or replaces multiple vectors. ids and vectors must have the same length.
func (*HNSW) Close ¶
Close persists any pending changes and releases resources. The index should not be used after Close.
func (*HNSW) Config ¶
func (h *HNSW) Config() HNSWConfig
Config returns a copy of the index configuration.
func (*HNSW) Flush ¶
Flush persists the index through the objectstore.ObjectStore if it has pending changes. For in-memory indexes (created via NewHNSW) this is a no-op.
func (*HNSW) Insert ¶
Insert adds or replaces a vector with the given ID. Returns an error if the vector dimension does not match the configured Dim.
func (*HNSW) Name ¶
Name returns the object name within the objectstore.ObjectStore for a persistent index, or "" for in-memory indexes.
func (*HNSW) Remove ¶
Remove removes the underlying object from the objectstore.ObjectStore. No-op for in-memory indexes. After Remove the index must not be used.
func (*HNSW) Save ¶
Save serializes the entire HNSW index to w in a compact binary format.
The format preserves internal node IDs so that neighbor references remain valid after deserialization. Deleted (free) slots are written as inactive markers to maintain index alignment.
Format overview:
[4B magic "HNSW"] [4B version]
[4B dim] [4B M] [4B efConstruction] [4B efSearch]
[4B numSlots] [4B activeCount] [4B maxLevel] [4B entryID]
[4B freeCount] [freeCount × 4B free IDs]
For each slot:
[1B active flag]
If active:
[4B idLen] [idLen bytes ID]
[4B level]
[dim × 4B float32 vector]
For each layer 0..level:
[4B numFriends] [numFriends × 4B friend IDs]
func (*HNSW) Search ¶
Search returns the top-k nearest vectors to the query, ordered by ascending distance (closest first).
func (*HNSW) SetEfSearch ¶
SetEfSearch adjusts the search-time candidate list size. Larger values improve recall at the cost of higher latency.
type HNSWConfig ¶
type HNSWConfig struct {
// Dim is the vector dimension. Required; must be positive.
// All inserted vectors must have exactly this many elements.
Dim int
// M is the maximum number of connections per node per layer (except
// layer 0, which allows 2*M). Higher values improve recall but
// increase memory usage and insertion time. Default: 16.
M int
// EfConstruction is the size of the dynamic candidate list during
// index building. Higher values produce a higher-quality graph at
// the cost of slower insertion. Default: 200.
EfConstruction int
// EfSearch is the default size of the dynamic candidate list during
// search queries. Higher values improve recall at the cost of higher
// latency. Can be adjusted at runtime via [HNSW.SetEfSearch].
// Default: 50.
EfSearch int
}
HNSWConfig configures a new HNSW index.
type HNSWLoadOptions ¶
type HNSWLoadOptions struct {
// MaxSlots caps the number of serialized slots. Zero means no explicit cap.
MaxSlots uint32
// MaxIDLen caps the serialized byte length of a node ID. Zero means no
// explicit cap.
MaxIDLen uint32
}
HNSWLoadOptions configures defensive bounds during deserialization.
type Index ¶
type Index interface {
// Insert adds or updates a vector with the given ID.
Insert(id string, vector []float32) error
// BatchInsert adds or updates multiple vectors at once.
// ids and vectors must have the same length.
BatchInsert(ids []string, vectors [][]float32) error
// Search returns the top-k nearest vectors to the query.
// Results are ordered by ascending distance (closest first).
Search(query []float32, topK int) ([]Match, error)
// Delete removes a vector by ID. No error if ID does not exist.
Delete(id string) error
// Len returns the number of vectors in the index.
Len() int
// Flush ensures all pending writes are visible to subsequent searches.
// For in-memory implementations this is typically a no-op.
Flush() error
// Close releases resources held by the index.
Close() error
}
Index is the interface for approximate nearest-neighbor search over dense float32 vectors.
All implementations must be safe for concurrent use.
type Match ¶
type Match struct {
// ID is the identifier of the matched vector.
ID string
// Distance is the distance between the query and matched vector.
// Lower values indicate higher similarity.
Distance float32
}
Match is a single result from a vector similarity search.
Source Files