Documentation
¶
Overview ¶
Package fdbdriver is a kine storage driver backed by FoundationDB.
Importing the package registers the "fdb" endpoint scheme with kine:
import _ "github.com/avkcode/kine-fdb/pkg/fdbdriver"
An endpoint names a FoundationDB cluster file and a namespace:
fdb:///etc/foundationdb/fdb.cluster?namespace=/clusters/prod-a
Each namespace is an independent etcd keyspace with its own revision sequence, stored in its own FoundationDB directory, so many Kubernetes control planes can share one FoundationDB cluster.
Every write is one FoundationDB transaction that takes the next revision, so revisions are dense and totally ordered within a namespace. Each change writes a history entry, the key's latest-state index row, and a change-log entry; watches follow the change log, and compaction deletes history that can no longer be read. See docs/architecture.md for the full layout.
Index ¶
- Constants
- Variables
- func New(ctx context.Context, wg *sync.WaitGroup, dc *drivers.Config) (bool, server.Backend, error)
- func NormalizeNamespace(ns string) (string, error)
- type Admin
- func (a *Admin) Compact(ctx context.Context, namespace string, revision, retain int64) (CompactStats, error)
- func (a *Admin) Delete(ctx context.Context, namespace string) error
- func (a *Admin) Namespaces(ctx context.Context) ([]string, error)
- func (a *Admin) Ping(ctx context.Context) error
- func (a *Admin) Stats(ctx context.Context, namespace string) (Stats, error)
- func (a *Admin) Verify(ctx context.Context, namespace string) (VerifyReport, error)
- type CompactStats
- type Config
- type Stats
- type TLSConfig
- type VerifyReport
Constants ¶
const ( // Scheme is the endpoint scheme this driver registers with kine. Scheme = "fdb" // DefaultNamespace is used when the endpoint does not name one. DefaultNamespace = "/kine/" // DefaultAPIVersion is the FoundationDB API version selected by default. // It is also the highest version the bundled Go bindings support. DefaultAPIVersion = 730 )
const ( // DirectoryRoot is the directory-layer path element under which every // namespace directory is created: ["kine", <namespace>]. DirectoryRoot = "kine" // StorageFormat is the on-disk layout version recorded in each namespace. StorageFormat = 1 )
Variables ¶
var ( // ErrNamespaceNotFound is returned by administrative operations on a // namespace that has never been created. ErrNamespaceNotFound = errors.New("fdb: namespace does not exist") )
Functions ¶
func New ¶
New is the kine driver constructor for the fdb scheme. FoundationDB is shared storage, so it asks kine embedders to enable leader election.
func NormalizeNamespace ¶
NormalizeNamespace returns the canonical "/a/b/" form of a namespace name. Leading, trailing, and repeated slashes are insignificant, so "tenant-a", "/tenant-a" and "/tenant-a/" all name the same namespace.
Types ¶
type Admin ¶
type Admin struct {
// contains filtered or unexported fields
}
Admin performs maintenance on namespaces without running a server.
func OpenAdmin ¶
OpenAdmin connects to the cluster named by endpoint. Methods that take a namespace use the endpoint's namespace when given "".
func (*Admin) Compact ¶
func (a *Admin) Compact(ctx context.Context, namespace string, revision, retain int64) (CompactStats, error)
Compact compacts a namespace to revision, or with retain > 0 to the current revision minus retain, and deletes everything that becomes unreachable. It also finishes any cleanup left by an interrupted run.
func (*Admin) Delete ¶
Delete removes a namespace and all of its data. A kine-fdb server still using the namespace fails its requests until it is restarted.
func (*Admin) Namespaces ¶
Namespaces lists every namespace in the cluster, sorted.
func (*Admin) Ping ¶
Ping checks that the cluster is reachable and the namespace exists, with a single small read. It is meant for readiness probes.
func (*Admin) Stats ¶
Stats scans a namespace. The scan spans many transactions, so on a busy namespace the counts are approximate.
func (*Admin) Verify ¶
Verify checks the invariants the driver maintains: every index row points at the newest history entry of its key, the change log is complete from the cleanup point to the current revision, chunked values are whole, and compaction left nothing behind that it should have removed. It holds the namespace's version map in memory, and concurrent writes can produce false reports, so run it on a quiet namespace.
type CompactStats ¶
type CompactStats struct {
// Revision is the compact revision after the run.
Revision int64 `json:"revision"`
// CurrentRevision is the namespace revision when the run started.
CurrentRevision int64 `json:"currentRevision"`
// Revisions is the number of change-log entries cleaned up.
Revisions int64 `json:"revisions"`
// DeletedVersions is the number of history entries removed.
DeletedVersions int64 `json:"deletedVersions"`
// Transactions is the number of cleanup transactions.
Transactions int `json:"transactions"`
}
CompactStats describes one compaction run.
type Config ¶
type Config struct {
// ClusterFile is the FoundationDB cluster file. Empty selects the client
// library default ($FDB_CLUSTER_FILE, ./fdb.cluster, then the platform path).
ClusterFile string
// Namespace isolates one etcd keyspace from others in the same cluster.
// It is normalized to the form "/name/".
Namespace string
// APIVersion is the FoundationDB API version to select.
APIVersion int
// PollInterval bounds how long watchers wait when a FoundationDB watch
// notification is missed. Changes normally arrive within milliseconds.
PollInterval time.Duration
// TxnTimeout bounds every transaction, including retries.
TxnTimeout time.Duration
// StartupTimeout bounds how long the driver waits for the cluster to
// become reachable when it starts.
StartupTimeout time.Duration
// TLS configures client TLS to the FoundationDB cluster. Empty fields fall
// back to the client library's FDB_TLS_* environment variables.
TLS TLSConfig
// TraceDir enables FoundationDB client trace files in this directory.
TraceDir string
// TraceFormat selects "xml" or "json" trace files.
TraceFormat string
// CompactInterval is the period of automatic compaction. Zero disables
// it, in which case compaction only happens on explicit requests.
CompactInterval time.Duration
// CompactIntervalJitter is a +/- percentage applied to CompactInterval.
CompactIntervalJitter int
// CompactTimeout bounds each compaction transaction.
CompactTimeout time.Duration
// CompactMinRetain is the number of most recent revisions never compacted.
CompactMinRetain int64
// CompactBatchSize is the number of revisions cleaned per transaction.
CompactBatchSize int64
// PollBatchSize is the maximum number of revisions read per watch poll.
PollBatchSize int64
}
Config is the fully resolved driver configuration.
func NewConfig ¶
NewConfig builds a Config from a kine driver configuration: the endpoint supplies FoundationDB settings, kine's flags supply compaction and polling settings, and kine's --ca-file/--cert-file/--key-file fill any TLS paths the endpoint leaves unset.
func ParseEndpoint ¶
ParseEndpoint parses an endpoint of the form
fdb://[cluster-file-path][?param=value&...]
The path after "fdb://" is the cluster file, so fdb:///etc/foundationdb/fdb.cluster names an absolute path and fdb://fdb.cluster a relative one. The cluster_file parameter overrides the path.
type Stats ¶
type Stats struct {
Namespace string `json:"namespace"`
Format int64 `json:"format"`
CurrentRevision int64 `json:"currentRevision"`
CompactRevision int64 `json:"compactRevision"`
CompactedRevision int64 `json:"compactedRevision"`
// Keys and DeletedKeys count index rows for live and deleted keys.
Keys int64 `json:"keys"`
DeletedKeys int64 `json:"deletedKeys"`
// LiveValueBytes is the total size of the current values of live keys.
LiveValueBytes int64 `json:"liveValueBytes"`
Versions int64 `json:"versions"`
ValueChunks int64 `json:"valueChunks"`
LogEntries int64 `json:"logEntries"`
// TotalKeyBytes and TotalValueBytes are the exact sizes of every
// FoundationDB key and value in the namespace.
TotalKeyBytes int64 `json:"totalKeyBytes"`
TotalValueBytes int64 `json:"totalValueBytes"`
// EstimatedBytes is FoundationDB's sampled size estimate, the figure
// reported to etcd clients as the database size.
EstimatedBytes int64 `json:"estimatedBytes"`
}
Stats describes the contents of a namespace.
type TLSConfig ¶
type TLSConfig struct {
CertFile string
KeyFile string
CAFile string
VerifyPeers string
Password string
}
TLSConfig holds FoundationDB client TLS settings.
type VerifyReport ¶
type VerifyReport struct {
Namespace string `json:"namespace"`
Checked int64 `json:"checked"`
Problems []string `json:"problems"`
// Truncated is set when more problems were found than reported.
Truncated bool `json:"truncated,omitempty"`
}
VerifyReport lists inconsistencies found by Verify.