skit

skit ([S]ervice [KIT]) Go SDK (software development kit) for building production web services that speak REST and/or gRPC, backed by Postgres, with first-class observability (slog + OpenTelemetry traces + Prometheus metrics), reliable background workers, and a transactional outbox.

OpenTelemetry compliant. Services built with skit follow the conventions of OpenTelemetry Go:

• traces are recorded with the OpenTelemetry Go SDK and exported over OTLP;
• the SDK is configured with the OTEL_* environment variables of the specification (OTEL_TRACES_EXPORTER, OTEL_TRACES_SAMPLER, OTEL_EXPORTER_OTLP_*, OTEL_RESOURCE_ATTRIBUTES);
• trace context is propagated as W3C TraceContext (traceparent, tracestate) over HTTP, gRPC and outbox/broker messages;
• server spans come from the official instrumentation (otelhttp, otelgrpc); span names, attributes and the Resource (service.*, deployment.environment.name, telemetry.sdk.*) follow the semantic conventions;
• every log record carries trace_id, span_id and trace_flags of its span.

Metrics are exposed in the Prometheus format.

This is NOT a framework. It is just a building block for production-ready applications in Go.

It distills patterns from several reference services into a single, importable module (github.com/assanoff/skit). A separate repo, skit-x, is the runnable showcase: a full CRUD app that imports this SDK from GitHub and exercises every package end-to-end (and is where the skit CLI is tried out).

The skit add generators scaffold per-model code: rest, rest-test, grpc, grpc-test, consumer, worker, migration, event, cron, and http-client. See quick-start.md for an end-to-end walkthrough.

Your code always imports the same path — github.com/assanoff/skit/... — regardless of how the source is supplied, because Go resolves packages by import path. So both recipes below produce identical runtime behavior; they differ only in versioning and workflow. The key rule for the local recipe: keep the module path github.com/assanoff/skit and wire it in with replace or a workspace — do not rename the import paths (that forks the SDK).

Versioned, reproducible, go.sum-verified. This is what skit new generates.

go get github.com/assanoff/skit@v0.6.0   # or @latest

import (
    "github.com/assanoff/skit/rest/router"
    "github.com/assanoff/skit/worker"
)

Update with go get github.com/assanoff/skit@latest. Pin a different version anytime by re-running go get …@vX.Y.Z.

Use this to develop your service and the SDK side by side, building against a local checkout. Two equivalent ways — pick one:

A. replace directive (per-module, committed to your app's go.mod):

# in your service module
go get github.com/assanoff/skit@v0.6.0          # keeps a sane version + go.sum entry
go mod edit -replace github.com/assanoff/skit=../skit
go mod tidy

// resulting go.mod
require github.com/assanoff/skit v0.6.0
replace github.com/assanoff/skit => ../skit

B. Go workspace (go.work, not committed — local to your machine):

cd /path/to/workspace        # parent dir holding both checkouts
go work init ./your-service ./skit

Either way the import paths stay github.com/assanoff/skit/... and the build uses the on-disk source. This is exactly how skit-x builds against the SDK during local development (replace github.com/assanoff/skit => ../skit).

Caveats for the local recipe:

• It builds against whatever is on disk — not version-pinned and not go.sum-verified for the replaced module; the path must exist everywhere you build (CI, teammates).
• replace/go.work are not transitive and not published — they apply only to your own build. Fine for an application (the main module); for a library you publish, ship the external dependency and drop the replace before tagging.
• Recommended flow: keep the external require pinned, add replace only while iterating on the SDK, then remove it.

• i18n.Translator loads JSON catalogs from an embed.FS, resolves the request language from Accept-Language, and localizes error responses: an app middleware translates any *errs.Error by its MessageID (or Code) with Args as template data. The example localizes widget errors to en/ru.
• auth verifies bearer tokens via a pluggable Verifier (built-in JWT covers HMAC, RSA/EC PEM, and JWKS through a custom Keyfunc) and enforces RBAC with Authenticate + RequireRole. The example protects widget writes behind a widget:write role while keeping reads public.

go run ./cmd/skit new github.com/you/svc          # embedded starter
go run ./cmd/skit new github.com/you/svc --template github.com/some/tmpl  # via gonew

See quick-start.md for an end-to-end walkthrough: scaffolding a service, running it, and generating REST/gRPC modules with the skit CLI.

broker + outbox give reliable, exactly-the-domain-write event publishing with a domain layer that knows nothing about SQL or the transport:

• The domain emits a plain typed value through a tx-bound outbox.Publisher:

  err := outbox.WithinTran(ctx, log, db, store, reg, func(tx *sqlx.Tx, pub outbox.Publisher) error {
      if err := repo.WithTx(tx).Create(ctx, w); err != nil { // domain write
          return err
      }
      return pub.Publish(ctx, widget.Created{ID: w.ID.String()}) // domain event
  })

  The event and the domain write commit in ONE transaction — an event is persisted iff its domain write commits. The domain names no exchange, topic, or routing key; it doesn't even hold a transaction.

• Routing is a wiring concern. A Registry, populated once at startup, maps each event's Go type to a transport-neutral Topic + optional Key:

  reg := outbox.NewRegistry()
  reg.Register[widget.Created]("widget.created", "widgets", outbox.WithKey("created"))

• outbox.Relay (a worker.Processor) drains pending events to any broker.Publisher with at-least-once delivery; Sweeper reclaims leases from crashed relays and Cleaner prunes terminal rows. The relay paces adaptively (worker.NewPacedLoop): a full batch drains again immediately, an empty one idles for PollInterval (optionally backing off to MaxPollInterval).

• Metrics are opt-in and registered on the app's shared registry: outbox.NewMetrics (relay/sweeper/cleaner throughput) + NewBacklogCollector (pending/in-flight/failed depth and oldest-pending age). SDK and business metrics never collide — distinct namespaces on one registry (see metrics).

• Each transport maps Topic/Key to its own concepts (RabbitMQ exchange+routing key via WithTopicMapper, Kafka topic+partition key, NATS subject), so the same event row can ship over any broker. broker/rabbitmq publishes CloudEvents v1.0 with publisher confirms and consumes them via a supervised worker.Runnable.

• Tracing flows end-to-end: Publish captures the W3C trace context into the event headers (otel.Carrier), the relay carries it onto the broker message, and the consumer restores it with otel.ExtractFromCarrier + opens a child span — so producing a widget and recording its audit row are one trace.

The skit-x showcase wires it end-to-end: creating a widget emits widget.Created through the outbox, the relay publishes it to RabbitMQ, and the widgetaudit consumer records it while continuing the producer's trace (see core/widget, widgetaudit, and the e2e test there, which asserts the consumer span is a child of the producer span).

worker, queue, and dbx compose into an at-least-once batch pipeline:

• worker.Pool — bounded-concurrency fan-out of one-shot jobs.
• worker.Processor[T] — a Source (claim) → Handler (process) → Sink (ack/retry) loop, run on a schedule via worker.Loop.
• queue.PG — a Postgres queue that hands each ready task to exactly one consumer via FOR UPDATE SKIP LOCKED, so N replicas drain it safely; it satisfies worker.Source[Task]/Sink[Task] directly.

The showcase wires this end-to-end: POST /widgets/import enqueues a batch, and a supervised import worker bulk-inserts it (idempotently, via dbx.BulkInsert + ON CONFLICT DO NOTHING). See skit-x/core/widgetimport.

A service usually starts as one process doing everything. When the HTTP side has to scale, the background loops must not scale with it — N replicas would poll the same rows N times and any wall-clock cron would fire N times.

The SDK deliberately ships no role/profile type for this: it already gives you two orthogonal gates, and the third question is answered by the command.

The scaffold (skit new --full) generates exactly that. A process is assembled like Lego from options (internal/app/server/options.go): server.New builds the dependency container and applies the options the command lists, app.Run supervises what they added. Each long-running command is that list:

// internal/cmd/worker.go
func (c *WorkerCommand) Execute(_ []string) error {
	return run(c.ServerOpts,
		server.WithServer(c.Debug.Addr, server.Debug),
		server.WithWorkers(),
	)
}

There is one kind of server: WithServer(addr, route sets…) — an httpserver.Server serving the route sets it is given. A "debug server" is not a type but a server with the technical sets: Health (/healthz, /readyz), Metrics, Pprof, Info (/debug/components), or Debug for all of them. Install is the business API and brings its own request middleware chain (trace, access log, HTTP metrics, timeout, body limit), so a set behaves the same on any server — probes and scrapes never reach the access log or the request timeout. Moving a set to another port is an edit of the command, not a config switch:

run(c.ServerOpts,
	server.WithServer(c.HTTP.Addr, server.Install),
	server.WithServer(":9100", server.Metrics),
	server.WithServer(c.Debug.Addr, server.Health, server.Pprof, server.Info),
)

No role type, no conditions inside the server package — what a process runs and where it listens is read off the command. Splitting a deployment is then a values-only change: the same image with a different command.

Every command keeps a server with Health: /healthz and /readyz are what keep a worker-only pod probeable and alive until SIGTERM, so one probe configuration fits every pod. GET /debug/components reports what the pod runs — the same values it logs at startup:

{"servers":[":8080",":6060"],"workers":0,"consumers":0}   ← http
{"servers":[":6060"],"workers":3,"consumers":0}           ← worker

For a cron that must fire once across the fleet, gate it with lock.Locker (skit add cron --lock postgres).

Why it is built this way, what a live split deployment showed (migration races, requests lost on rollout, the release layout) and what is still open — see research-pod-split.md.

grpcserver runs alongside (or instead of) the REST server — both are worker.Runnables supervised by worker.Group, toggled independently by config. It ships:

• a server span per RPC (otelgrpc, Config.TracerProvider) and the same cross-cutting interceptors as REST (panic recovery, structured access logs, Prometheus metrics) plus automatic *errs.Error → gRPC status mapping (error codes are aligned, so the mapping is a direct cast);
• messages use the protobuf Opaque API via Protobuf Editions (edition = "2023", api_level = API_OPAQUE) — Google's official path to faster, lower-allocation, lazy-decoding generated code (replaces the third-party vtprotobuf; no extra dependency or codec);
• transport tuning (MaxRecvMsgSize, NumStreamWorkers, keepalive);
• gRPC health service and optional reflection.

Code is generated with buf v2 (config lives with the app in skit-x). gRPC v1.84 / protobuf v1.36.

make proto-tools   # install buf, protoc-gen-go, protoc-gen-go-grpc
make proto         # buf lint + generate in ../skit-x (proto -> gen)

make build      # build the SDK
make test       # unit tests
make check      # everything CI checks: tidy, go fix, lint, govulncheck, tests

See skit-x for a full application wired with dim (go run ./cmd serve), plus its integration tests (make test-integration).