engine

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 22, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var DefaultFlowConcurrency = 0

DefaultFlowConcurrency bounds how many flow instances may be mid-flight at once. Zero means unbounded -- the default, because bounding is an observable change: at saturation the pool must either block issuance (actual flow rate drops below configured), drop work, or queue unboundedly, so opting into a bound is explicit. When bounded, this implementation BLOCKS and counts the event, visible in the snapshot as Saturated rather than silent. At rate=50 with multi-second step delays the steady state is a few hundred instances.

Functions

This section is empty.

Types

type BrokerController

type BrokerController interface {
	StopBroker(ctx context.Context, name string) error
	StartBroker(ctx context.Context, name string) error
}

BrokerController stops and starts brokers.

Two real implementations -- the local Docker cluster and a no-op for external clusters -- which is exactly the bar for defining an interface.

type Config

type Config struct {
	Kafka     kafka.Config
	Scenarios []*scenario.Scenario

	// Duration of zero means run until the context is cancelled.
	Duration time.Duration

	NoConsumers    bool
	RecreateTopics bool

	// SkipTopicCreation leaves topic management entirely alone: nothing is created and
	// nothing is deleted.
	//
	// It exists for clusters where khaos has no CreateTopics permission, which is the
	// normal case on a managed cluster, so conflating it with RecreateTopics (only
	// skipping the DELETE half) leaves exactly that user failing at startup on the
	// CreateTopics call.
	SkipTopicCreation bool

	// Seed makes generated data reproducible. A zero Seed means "pick one", and the
	// chosen value is reported so a run can be replayed.
	Seed uint64

	// Brokers is nil for external clusters, where broker faults are unavailable.
	Brokers BrokerController

	// Registry is the Schema Registry endpoint and its credentials.
	//
	// A bare URL is enough for the bundled local registry; Confluent Cloud and Aiven
	// require auth. An empty Registry.URL falls back to the scenario's own
	// `schema_registry:` block.
	Registry codec.RegistryConfig

	// LagPoll is how often to ask the brokers for real consumer-group lag. Zero disables
	// polling, which is the default.
	//
	// The self-reported Lag figure is produced-minus-consumed from khaos's own in-process
	// counters -- wrong the moment anything else touches the topic, and meaningless
	// across restarts. Real lag needs admin traffic and DESCRIBE rights, which a managed
	// cluster may not grant, so it is opt-in and MUST degrade to "unknown" rather than
	// failing the run.
	LagPoll time.Duration

	Logger *slog.Logger

	// Metrics is where per-message Prometheus counters are recorded, keyed by topic and
	// group as telemetry.Metrics defines. Nil disables it entirely -- every call site
	// checks before touching it, so an engine built without --metrics-addr pays nothing
	// beyond the nil check.
	Metrics *telemetry.Metrics
}

Config is everything the engine needs to run.

type Consumer

type Consumer struct {
	ID      scenario.ID
	Topic   string
	GroupID string
	Topics  []string
	Conf    scenario.ConsumerConf
	// contains filtered or unexported fields
}

Consumer polls one topic as part of a consumer group, optionally simulating failures.

func (*Consumer) Close

func (c *Consumer) Close()

Close releases the underlying client, leaving the group cleanly.

func (*Consumer) Pause

func (c *Consumer) Pause()

Pause stops consuming without tearing the consumer down.

func (*Consumer) Paused

func (c *Consumer) Paused() bool

Paused reports whether this consumer is currently parked.

func (*Consumer) Resume

func (c *Consumer) Resume()

Resume unblocks a paused consumer; the original goroutine simply continues.

func (*Consumer) Run

func (c *Consumer) Run(ctx context.Context) error

Run consumes until ctx is cancelled.

func (*Consumer) SetDelay

func (c *Consumer) SetDelay(ms int)

SetDelay changes the artificial processing delay live.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine owns a run: every producer, consumer, the incident scheduler, and the counters they feed.

The engine has exactly one read API -- Snapshot -- and knows nothing about terminals, colours or rendering. The duration deadline and shutdown trigger live in the engine itself, never in an output loop.

func New

func New(ctx context.Context, cfg Config) (*Engine, error)

New builds an engine: it creates topics, constructs codecs and clients, and registers every producer and consumer. It performs I/O and can fail.

Construction is deliberately separate from Run so that a configuration or connectivity problem surfaces before any traffic is generated and before any lifecycle exists to cancel it.

func (*Engine) Healthy

func (e *Engine) Healthy() error

Healthy reports whether the engine is in a state it can recover from. It backs /healthz.

func (*Engine) Run

func (e *Engine) Run(ctx context.Context) error

Run drives the scenario until the duration expires or ctx is cancelled, then tears everything down deterministically.

Two structural properties matter here:

  1. The run deadline lives in a context, not in the render loop, so a blocking render can never starve the signal handler and make an infinite run unstoppable.

  2. The output layer is NOT a member of this errgroup and never appears here at all. The caller runs it separately against Snapshot. If it were a member, a TUI panic would cancel the group and kill the engine -- the same coupling in fail-fast form.

func (*Engine) Seed

func (e *Engine) Seed() uint64

Seed reports the seed this run used, so a run can be reproduced.

func (*Engine) Snapshot

func (e *Engine) Snapshot() Snapshot

Snapshot returns a fully-owned view of current engine state.

This is the engine's entire read API. Everything returned is a value or a freshly allocated slice, so a consumer may hold it indefinitely without racing the engine. Consumers PULL on their own schedule rather than the engine pushing: a wedged UI then produces stale numbers instead of backpressure into the run.

type Event

type Event struct {
	At      time.Time
	Message string
	Level   EventLevel
}

Event is something noteworthy that happened during a run.

Events accumulate in a bounded ring and are read through Snapshot; the engine never writes to a terminal directly, so the output layer decides whether an event becomes a log line, a TUI row, or nothing at all.

type EventLevel

type EventLevel = scenario.EventLevel

EventLevel mirrors scenario.EventLevel for output consumers that should not need to import the scenario package.

type FlowStat

type FlowStat struct {
	Name      string
	Started   int64
	Completed int64
	Messages  int64
	Errors    int64

	// InFlight is the number of flow instances currently executing their step delays.
	// The pool is bounded and this depth is reported live.
	InFlight  int64
	Saturated int64 // times issuance blocked because the pool was full
}

FlowStat is the per-flow slice of a Snapshot.

type GroupStat

type GroupStat struct {
	GroupID   string
	Consumers int
	Consumed  int64
	Failed    int64
	DLQ       int64
	Paused    int
}

GroupStat is one consumer group's contribution to a topic.

type Producer

type Producer struct {
	ID    scenario.ID
	Topic string
	// contains filtered or unexported fields
}

Producer drives one stream of generated records at a controlled rate.

One goroutine, calling the Kafka client directly -- no worker pool and no ceiling on concurrent producers.

func (*Producer) Inflight

func (p *Producer) Inflight() int64

Inflight reports records handed to the client but not yet acknowledged.

func (*Producer) Rate

func (p *Producer) Rate() float64

Rate reports the current target rate, with +Inf meaning unlimited.

func (*Producer) Run

func (p *Producer) Run(ctx context.Context) error

Run produces until ctx is cancelled.

Returns nil on cancellation: a cancelled run is a successful run, not an error. Only a genuine generator failure propagates, because that means the scenario itself is unusable and every other producer will hit the same thing.

func (*Producer) SetRate

func (p *Producer) SetRate(perSecond float64)

SetRate retunes the running producer to a new target rate, live. Relies on the per-producer limiter above so a targeted rate change retunes only the targeted producer, not every producer on its topic.

type Snapshot

type Snapshot struct {
	At       time.Time
	Elapsed  time.Duration
	Scenario string

	// Deadline is the configured run duration; zero means run until interrupted.
	Deadline time.Duration

	Topics []TopicStat
	Flows  []FlowStat
	Events []Event

	TotalProduced int64
	TotalConsumed int64
	TotalErrors   int64
	Rebalances    int64

	// Healthy is false when the engine has recorded a condition it cannot recover from
	// on its own. It drives /healthz.
	Healthy  bool
	Stopping bool
}

Snapshot is a consistent-enough, fully-owned view of engine state.

This is the engine's ENTIRE read API. The TUI, the headless log loop, the Prometheus collector and any future web UI are all consumers of this one method. Everything in it is a value or a freshly allocated slice: a consumer can hold a Snapshot indefinitely without racing the engine or keeping engine memory alive.

type TopicStat

type TopicStat struct {
	Topic    string
	Scenario string

	Produced   int64
	Consumed   int64
	Bytes      int64
	ProduceErr int64
	ConsumeErr int64
	Duplicates int64

	// CommitErr counts offsets khaos declined to commit or failed to commit.
	CommitErr int64

	// Failed, DLQ and Retries are only meaningful when failure simulation is enabled.
	Failed  int64
	DLQ     int64
	Retries int64

	// Lag is produced-minus-consumed from khaos's own counters -- a self-report, not
	// Kafka consumer lag. It is meaningless across restarts and wrong whenever anything
	// else produces to or consumes from the topic. It is always populated because it
	// costs nothing; BrokerLag is the real answer when the cluster will give one.
	Lag int64

	// BrokerLag is real committed-offset-vs-end-offset lag per group, summed over the
	// topic's partitions and populated only when lag polling is enabled (Config.LagPoll).
	//
	// Nil means "not measured" and MUST be rendered as unknown rather than zero: polling
	// off, the group not created yet, and a cluster that denies DESCRIBE on consumer
	// groups all land here, and none of them means the group has caught up. A group
	// missing from a non-nil map is unknown for the same reason -- the other groups
	// answered and it did not.
	BrokerLag map[string]int64

	Groups []GroupStat
}

TopicStat is the per-topic slice of a Snapshot.

Everything measured is exposed here, including bytes, errors, duplicates, retries and commit failures; the output layer chooses what to render.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL