Documentation
¶
Overview ¶
Package postgreslog is a Postgres-backed govern.EventLog for HIGH-AVAILABILITY governed state: multiple processes read the same governed event stream and replay it to the same convergent state. gsm guarantees order-independent convergence (WFC + CC), so shared-log replay is consistent across nodes regardless of the exact order each observes. It parallels store/postgres for the durable journal. It satisfies govern.EventLog structurally, so this package does not import govern, keeping the coupling one-directional.
An append is one INSERT statement sent on its own, which Postgres runs as a transaction of its own and commits before it replies. Its position comes from the function governed_events_next_seq_v1, which takes the entity's transaction-level advisory lock and then reads MAX(seq)+1, so appends to one entity queue on the lock instead of racing for a position (see Append). No transaction spans two round trips, so a process that stops between two of its round trips (a SIGSTOP, a suspended VM, a long GC pause, a partition) holds no lock another process's append waits on. Reads are single SELECT statements. Each statement behaves the same whatever the deployment's default_transaction_isolation: one that fails with a serialization failure (40001), which repeatable read or serializable reports where read committed would act on the latest row, changed nothing and is run again. The schema migration is the one transaction of several statements; it sets read committed itself (see txOptions).
Every name in every statement the log sends is qualified: governed_events and next_seq with the log's schema, and every function, type and operator with pg_catalog (operators written OPERATOR(pg_catalog.<op>)), so no statement, Open's and the migration's included, looks a name up through the search path (the statement check in sqlcheck.go holds them to it). Which schema is the log's is the one thing the search path can still decide, and only when the schema is not pinned. With WithSchema, the recommended deployment, the search path plays no part. Without it, every Open discovers the schema: the first schema on the search path holding a relation named governed_events, or the first schema on the path when none does yet (see logSchema); it refuses a first relation of that name that is not an ordinary table, with no partitions or inheritance, migrates a legacy table in a later schema in place, and logs a warning. Discovery runs again at every Open, so a role that can create a schema earlier on the path (any role with CREATE on the database can create the "$user" schema the default search path puts first) can redirect a restarting process: discovery is safe only when every schema on the search path is trusted. Within one process, after Open, no role can redirect the log, pinned or not.
The log trusts the owner of its schema and every role that can create objects in it, as it trusts the table's owner: such a role could replace the table. Within that boundary it refuses what it can check: the next_seq function must be owned by the table's owner, run with search_path = pg_catalog, pg_temp and have exactly this version's body, whose names are all qualified. Unpinned, the log also trusts every role that can create a schema earlier on its search path (see above). Any role that can connect can hold an entity's advisory key (pg_advisory_lock(hashtextextended(entity, 0))) and stall its appends, as in earlier versions; it corrupts nothing. After ALTER SCHEMA ... RENAME, drop the function (DROP FUNCTION <schema>.governed_events_next_seq_v1(text)) and Open again.
The migration also records the schema version in governed_events_schema_version (1). A log created before that table existed is version 1; the migration adds the table. Open refuses another version with ErrSchemaVersion.
A retry does not always follow another transaction's commit: at serializable, Postgres may fail a statement for a conflict with a transaction that has not committed yet, and many processes appending to one entity may each lose their position several times. So the log spaces the attempts with a capped, jittered exponential backoff (from 1ms up to 100ms, a random fraction of it each time) and keeps retrying until the statement succeeds, fails for another reason, or ctx is done: a bound on the attempts would turn contention into errors. A caller that needs a bound on the time an append may take sets a deadline on ctx; when ctx ends, the call returns an error that wraps both ctx's error and the last attempt's.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrSchemaMissing = errors.New("postgreslog: the schema is missing")
ErrSchemaMissing is wrapped by the error Open returns when the log's schema (the pinned schema, governed_events with its append_id index, or the next_seq function) does not exist and the log was opened WithoutMigration. Run Migrate as the schema's owner, then Open again. The error also wraps agent.ErrConfig. It is the counterpart of store/postgres's ErrSchemaMissing; this package does not import that module.
var ErrSchemaVersion = errors.New("postgreslog: the schema version does not match")
ErrSchemaVersion is wrapped by the error returned when the log's recorded schema version is not the one this version of the package uses: a newer one needs a newer program, and there is no migration from another one. The error also wraps agent.ErrConfig. It is the counterpart of store/postgres's ErrSchemaVersion.
Functions ¶
func Migrate ¶ added in v0.17.0
Migrate opens the log at dsn as the role in the DSN, which needs the privileges of the schema's owner, creates or migrates governed_events and the next_seq function as Open does, checks them, and closes the pool. It is idempotent. Run it as the owner, and have the service Open the log WithoutMigration. WithoutMigration is an agent.ErrConfig error here.
Types ¶
type Log ¶
type Log struct {
// contains filtered or unexported fields
}
Log is a Postgres-backed append-only event log per entity.
func Open ¶
Open connects to Postgres via a pgx DSN (e.g. "postgres://user:pass@host:5432/db") and ensures the schema exists, or with WithoutMigration only verifies it. Pin the schema with WithSchema; without it, Open discovers the schema through the search path (see logSchema) and logs a warning.
func (*Log) Append ¶
Append durably records an event for an entity under id and returns its position (its seq). Concurrent appends to the same entity, from any number of processes, take dense and unique positions in commit order, so the entity's log is append-only as every reader sees it. If the entity already holds id, nothing is inserted and the recorded position is returned.
The append is one INSERT, committed before Postgres replies, that takes the next position (MAX(seq)+1) from nextSeqFunction, which first takes the entity's lock and so queues appends to one entity. An append computes position n+1 only when the event at n is committed and visible to it, so no reader sees an event before one at a lower position. Should two appends still read the same MAX (at repeatable read or serializable, or beside a writer that does not take the lock), they collide on the primary key (entity, seq): the later one waits for the first to commit, fails with 23505, and runs again with a snapshot that sees it. An append whose id is already recorded meets the (entity, append_id) index and does nothing, once the recording append has committed.
An append whose commit reply is lost is not retried here: database/sql retries a statement only on driver.ErrBadConn, which the driver reports only when nothing was sent on the connection. A caller that retries after such an error sends the same id, which returns the recorded position.
func (*Log) Events ¶
Events returns an entity's events at positions from onward, in log order. Positions are dense, so the i-th event read is the one at position from+i; a row whose seq breaks that (a row deleted or renumbered outside this adapter) is an error wrapping agent.ErrProtocol, since the caller would otherwise take a later event for the one at the missing position.
type Option ¶ added in v0.9.0
type Option interface {
// contains filtered or unexported methods
}
Option configures Open.
func WithSchema ¶ added in v0.9.0
WithSchema pins the schema governed_events and its next_seq function are in, instead of discovering it through the search path at every Open. It is the recommended deployment: with it, the search path plays no part in which schema the log uses, and no role can redirect a restarting process by creating a schema earlier on the path. The name is used as given (quoted), so "App" and "app" are different schemas, and Postgres truncates it, like any identifier, to 63 bytes. The schema must exist: Open fails with ErrConfig otherwise, and never creates it. An empty name, and a system schema (information_schema, or a name starting with pg_, pg_temp among them, which names each session's own temporary schema), is an ErrConfig error.
func WithoutMigration ¶ added in v0.17.0
func WithoutMigration() Option
WithoutMigration makes Open verify the schema instead of creating it: it runs no CREATE and no ALTER, so its role needs no CREATE on the schema. The owner runs Migrate once at deploy time. Open refuses a schema that lacks governed_events (with its append_id index) or the next_seq function with an error that wraps ErrSchemaMissing and agent.ErrConfig, and checks the unique indexes and the function as it does after a migration. The role needs USAGE on the schema, SELECT and INSERT on governed_events and EXECUTE on the next_seq function.
Source Files