ceph

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 16, 2026 License: MIT Imports: 48 Imported by: 0

Documentation

Overview

Package ceph provisions disposable Ceph clusters using testcontainers-go. It controls Ceph through container-local CLIs without linking host librados.

Run starts the MON, MGR and OSD topology with cluster-wide options: daemon counts, OSD storage and placement, networks, pools and Messenger mode. Pool policies, Cephx identities, configuration overrides, fault injection and health observations operate on the returned Container.

Client services live in their own packages, which add their options to the same Run and declare their own service types:

  • github.com/JSYoo5B/ceph-testcontainers-go/rbd: RBD pools, namespaces, mirroring and backup.
  • github.com/JSYoo5B/ceph-testcontainers-go/cephfs: filesystems, MDS topology, subvolumes and mirroring.
  • github.com/JSYoo5B/ceph-testcontainers-go/rgw: gateways, users, placement and multisite replication.

Each service package has its own Run that starts a cluster with that service's defaults. Options from several service packages can be passed to one ceph.Run to start a cluster that serves them together.

This is an experimental module for application integration tests.

Index

Examples

Constants

View Source 
const (
	MaxNativeCommandTimeout = time.Hour
	MaxNativeCommandOutput  = 64 << 20
	MaxNativeCommandInput   = 64 << 20
	// NativeCommandStopTimeout bounds one attempt to stop a command inside its
	// container: the canceler and, when it fails, one inspection of the
	// container. The attempt has this budget of its own, independent of the
	// caller's context, which has usually ended.
	NativeCommandStopTimeout = supervised.CancelTimeout + nativeInspectTimeout
)

Limits of a NativeCommand.

View Source 
const CrashDir = "/var/lib/ceph/crash"

CrashDir is the crash directory of every daemon of the cluster. The bootstrap ceph.conf sets it, and WithConfigFile refuses to change it.

View Source 
const DefaultImage = "quay.io/ceph/ceph:v20.2.4@sha256:6bb1c8a42fbc0bf87938946990b65174466997bc11c31eb5a323225a779fd8f9"

DefaultImage pins the multi-architecture manifest tested by this PoC.

View Source 
const MaxNativeFailureReason = 512

MaxNativeFailureReason bounds NativeFailure.Reason.

View Source 
const MaxNetworkDelay = 10 * time.Second

MaxNetworkDelay bounds NetworkDelay.Delay.

View Source 
const NetemHelperImage = "ghcr.io/jsyoo5b/ceph-testcontainers-images:extension-netem-v1"

NetemHelperImage is the netem helper that ceph-testcontainers-images publishes for contract 1 of its netem extension, for linux/amd64 and linux/arm64. TemporaryNetworkDelay never selects it by itself: pass it, or another image that meets the contract, as NetworkDelay.HelperImage.

Variables

View Source 
var (
	// ErrInvalidRequest reports a request that the fixture refused before
	// any change: malformed arguments, or an operation that the state of the
	// cluster or of the selected resource forbids, such as a terminated
	// cluster, a container that the cluster neither owns nor has attached,
	// or an override that is already held.
	ErrInvalidRequest = errors.New("invalid request")
	// ErrUnsupported reports a native feature that the Ceph of the relevant
	// target lacks, established before any change from that target's own
	// capability, such as an option that its tool does not list, or from a
	// documented limit of the release it runs. *OperationError names the
	// feature, the target and its version.
	ErrUnsupported = errors.New("unsupported by the target's Ceph")
	// ErrImageCapability reports an image that lacks what the operation
	// needs: an executable or runtime, such as the python3 of the
	// supervisor, or an optional extension that was not selected or does not
	// meet its contract.
	ErrImageCapability = errors.New("image capability unavailable")
	// ErrEnvironmentUnavailable reports that the Docker engine, the host, the
	// kernel or the permissions of a container cannot provide what the
	// operation needs, although the image may be valid. When it comes from a
	// call that had sent a native command, the command may have run.
	ErrEnvironmentUnavailable = errors.New("runtime environment unavailable")
	// ErrNativeFailure reports a native command that ran and failed: Ceph
	// refused the request, a resource was missing, permission was denied or
	// I/O failed. *NativeFailure carries the native status.
	ErrNativeFailure = errors.New("native operation failed")
)

Error categories. A classified error matches one of them with errors.Is, and errors.As with *OperationError or *NativeFailure returns its details. An error joined from several failures, such as a failed change and a failed restoration, can match several. Context errors, ErrNativeCommandIncomplete and *NativeCommandUnstoppedError keep their own contracts and belong to no category: a call that ctx ended is never classified by what it could not finish. An error that matches no category is a failed check of the fixture's own, such as output it could not parse, or a failure whose outcome is unknown, such as a container that disappeared after a native command was sent to it. Only ErrInvalidRequest promises that nothing changed.

View Source 
var (
	// ErrNetemImageUnsupported reports a helper image that does not meet the
	// netem contract: a missing or broken tc, ip, sh, sleep or python3, or a
	// tc without netem. It belongs to ErrImageCapability.
	ErrNetemImageUnsupported error = &categoryError{text: "the netem helper image does not meet the netem contract", kind: ErrImageCapability}
	// ErrNetemPermissionDenied reports an engine that did not give the helper
	// NET_ADMIN, or refused a qdisc change for lack of it. It belongs to
	// ErrEnvironmentUnavailable.
	ErrNetemPermissionDenied error = &categoryError{text: "the netem helper has no NET_ADMIN", kind: ErrEnvironmentUnavailable}
	// ErrNetemKernelUnsupported reports a kernel without the netem qdisc. It
	// belongs to ErrEnvironmentUnavailable.
	ErrNetemKernelUnsupported error = &categoryError{text: "the engine's kernel has no netem qdisc", kind: ErrEnvironmentUnavailable}
)
View Source 
var ErrHostNetworkUnavailable error = &categoryError{text: "ceph host network is unavailable to the test process", kind: ErrEnvironmentUnavailable}

ErrHostNetworkUnavailable reports that host mode cannot serve native clients in the test process: the engine does not run Linux containers, the selected address is not assigned on the Docker host, or the first MON's advertised endpoint is not reachable from this process. Run returns it before creating MGR/OSD daemons; later MON/RGW port reservations may also return it. It belongs to ErrEnvironmentUnavailable.

View Source 
var ErrNativeCommandIncomplete = errors.New("native command did not complete within its limits")

ErrNativeCommandIncomplete marks a NativeCommandResult that timed out or whose output was truncated: its bytes are not the command's complete output.

View Source 
var ErrNativeInputClosed = errors.New("native command input is closed")

ErrNativeInputClosed reports a write to a NativeInput that was closed, or whose command no longer takes input.

Functions

func WithIdleEntrypoint 

func WithIdleEntrypoint() testcontainers.CustomizeRequestOption

WithIdleEntrypoint keeps a client or helper container running with sleep infinity under Docker's init process, so commands can be executed in it later. As PID 1, sleep ignores SIGTERM and Docker waits for the full stop timeout (10 seconds by default) before killing it; init forwards the signal so termination finishes at once. Init also reaps background processes that exit inside the container, such as a test server started through Exec. The entrypoint and command remain sleep and infinity.

Types

type AddressFamily added in v0.2.0 

type AddressFamily int

AddressFamily selects the addresses that the Messenger of every daemon binds and advertises. 

const (
	// AddressIPv4 binds and advertises the containers' IPv4 addresses.
	AddressIPv4 AddressFamily = iota
	// AddressIPv6 runs the cluster on a dual-stack bridge and binds and
	// advertises only the containers' IPv6 addresses.
	AddressIPv6
	// AddressIPv4MappedIPv6 keeps an IPv4 bridge and binds and advertises the
	// containers' IPv4 addresses as IPv4-mapped IPv6 addresses
	// (::ffff:a.b.c.d) on IPv6 sockets.
	AddressIPv4MappedIPv6
)

func (AddressFamily) String added in v0.2.0 

func (family AddressFamily) String() string

type AuthConfig added in v0.2.0 

type AuthConfig struct {
	// KeyType is the type of the keys the fixture generates itself: the
	// mon., client.admin and OSD keys and, unless WithClientKeyType selects
	// another, CreateClient keys.
	KeyType CephxCipher
	// ServiceCipher encrypts service tickets. AllowedCiphers are the key
	// types that the MONs accept for authentication, and PreferredCipher is
	// the type the MONs use for keys they generate with auth get-or-create,
	// such as MGR, MDS and RGW keys. These MonMap settings need a KeyType and,
	// for anything but CipherAES, the image support that CipherAES256K
	// describes. ServiceCipher and PreferredCipher
	// default to KeyType, and AllowedCiphers to the distinct types among
	// KeyType, ServiceCipher and PreferredCipher. AllowedCiphers must contain
	// all three.
	ServiceCipher   CephxCipher
	AllowedCiphers  []CephxCipher
	PreferredCipher CephxCipher
	// MonTicketTTL and ServiceTicketTTL set auth_mon_ticket_ttl and
	// auth_service_ticket_ttl, in seconds with fractions kept, such as
	// 10500 * time.Millisecond. MONs rotate service secrets on their ticks, so
	// lifetimes of a few seconds also need a shorter mon_tick_interval, which
	// WithConfigFile can set. Keep MonTicketTTL at ten seconds or more: MON
	// clients renew their tickets only every few seconds, and a daemon whose
	// renewal comes too late is refused and does not authenticate again.
	MonTicketTTL     time.Duration
	ServiceTicketTTL time.Duration
}

AuthConfig selects the Cephx settings of a new cluster. Zero fields keep the defaults of the image's Ceph release.

type BlocklistEntry 

type BlocklistEntry struct {
	Address string
	Range   bool
	Until   time.Time
}

BlocklistEntry reports native OSDMap fencing, including ranges created by external tools. Until is the MON's absolute expiration time. A listing grants no ownership to remove an existing entry.

type BlocklistOverride 

type BlocklistOverride struct {
	// contains filtered or unexported fields
}

BlocklistOverride owns one newly added exact client-session entry. Address includes a nonzero nonce, so two clients on one host remain distinct. Copies share restoration state. Removing the entry restores OSDMap admission; session recovery depends on the client library and callers may need to reconnect. External blocklist edits must not race helpers.

func (*BlocklistOverride) Address 

func (change *BlocklistOverride) Address() string

func (*BlocklistOverride) Restore 

func (change *BlocklistOverride) Restore(ctx context.Context) error

Restore removes only this newly created exact entry. Prior native entries and ranges are preserved. A changed expiration is an outside edit and is refused. Lost removal replies can be retried; after confirmed absence copies are no-ops.

type CephxCipher added in v0.2.0 

type CephxCipher string

CephxCipher names a Cephx key type, which is also the cipher of the tickets made with such keys. 

const (
	// CipherAES is AES-128-CBC, the only key type of Squid and of Tentacle
	// before 20.2.4.
	CipherAES CephxCipher = "aes"
	// CipherAES256K is AES-256 with Kerberos-style key derivation
	// (AES256KRB5). It needs an image whose ceph-authtool has --key-type and
	// whose monmaptool has the cipher options, as in Ceph 20.2.4 and later,
	// where it is the default.
	CipherAES256K CephxCipher = "aes256k"
)

type ClientCaps 

type ClientCaps struct {
	Mon string
	OSD string
	MGR string
	MDS string
}

ClientCaps contains Cephx capability expressions, passed as individual CLI arguments. For example, Mon: "allow r" and OSD: "allow rw pool=data namespace=tenant-a" restrict RADOS I/O to one pool and namespace. Empty fields grant no capabilities for that service. Capabilities are never broadened by this helper; Ceph validates their syntax.

type ClientConfig 

type ClientConfig struct {
	// contains filtered or unexported fields
}

ClientConfig describes a client identity created by this cluster. Credentials remain private; ConnectionConfig returns copies for an external native client. The cluster owns the auth database, while callers own attached containers.

func (*ClientConfig) ConnectionConfig 

func (client *ClientConfig) ConnectionConfig() ([]byte, []byte, error)

ConnectionConfig returns independent ceph.conf and keyring copies. The config is a snapshot at creation; WithClientIdentity uses the current MON addresses. Revocation does not erase already copied credentials or invalidate established sessions immediately. Test revoked credentials with a fresh connection.

func (ClientConfig) GoString 

func (client ClientConfig) GoString() string

GoString excludes credentials from formatted Go representations as well.

func (*ClientConfig) KeyringPath 

func (client *ClientConfig) KeyringPath() string

KeyringPath returns the per-identity path used in an attached client container.

func (*ClientConfig) Name 

func (client *ClientConfig) Name() string

Name returns the fully qualified Cephx entity, for example client.reader.

func (ClientConfig) String 

func (client ClientConfig) String() string

String deliberately excludes credentials, including when a descriptor is logged.

func (*ClientConfig) User 

func (client *ClientConfig) User() string

User returns the ID without the client. prefix, for APIs such as librados rados_create and go-ceph rados.NewConnWithUser.

type ClientContainerOption added in v0.2.0 

type ClientContainerOption func(*clientContainerOptions) error

ClientContainerOption customizes the configuration that WithClient and WithClientIdentity write into a client container.

func WithClientMessengerPolicy added in v0.2.0 

func WithClientMessengerPolicy(policy ClientMessengerPolicy) ClientContainerOption

WithClientMessengerPolicy writes the policy into the [client] section of the client container's ceph.conf. It applies only to the Ceph clients in that container, overrides the central configuration and MessengerV2Secure's global modes, and leaves the cluster's own control CLI alone.

type ClientMessengerPolicy added in v0.2.0 

type ClientMessengerPolicy struct {
	// MonitorsAndManagers sets ms_mon_client_mode, which Ceph uses for
	// connections to MONs and MGRs.
	MonitorsAndManagers []MessengerConnectionMode
	// OtherDaemons sets ms_client_mode, which Ceph uses for connections to
	// the other daemons, such as OSDs.
	OtherDaemons []MessengerConnectionMode
}

ClientMessengerPolicy selects the Messenger v2 connection modes that a native client offers, in order of preference. A nil field keeps Ceph's default, which offers both modes.

type ClientOption added in v0.2.0 

type ClientOption func(*clientOptions) error

ClientOption customizes CreateClient.

func WithClientKeyType added in v0.2.0 

func WithClientKeyType(cipher CephxCipher) ClientOption

WithClientKeyType selects the key type of a new client identity instead of the cluster's AuthConfig.KeyType or the release default. The MONs must allow the type for the client to authenticate. CipherAES256K needs an image with key type support (see CipherAES256K); without it CreateClient fails before any change.

type ClientRevocation added in v0.2.0 

type ClientRevocation struct {
	// contains filtered or unexported fields
}

ClientRevocation is a temporary removal of a client identity that this cluster created; Restore puts back the same key and capabilities. It holds the original credentials privately and never prints them. Copies of a revocation share its state.

func (ClientRevocation) GoString added in v0.2.0 

func (r ClientRevocation) GoString() string

GoString excludes credentials from formatted Go representations as well.

func (*ClientRevocation) Name added in v0.2.0 

func (r *ClientRevocation) Name() string

Name returns the revoked entity, for example client.reader.

func (*ClientRevocation) Restore added in v0.2.0 

func (r *ClientRevocation) Restore(ctx context.Context) error

Restore puts back the identity's original key and capabilities and makes its descriptor usable again. It succeeds again once restored, as long as the identity still holds them. It refuses an identity that was created again with another key or other capabilities, and one that DeleteClient removed for good. The restoration is fenced like the removal: when its outcome stays uncertain, the next Restore settles first.

func (ClientRevocation) String added in v0.2.0 

func (r ClientRevocation) String() string

String deliberately excludes credentials.

type ConfigEntry 

type ConfigEntry struct {
	Section            string `json:"section"`
	Mask               string `json:"mask"`
	Name               string `json:"name"`
	Value              string `json:"value"`
	Level              string `json:"level"`
	CanUpdateAtRuntime bool   `json:"can_update_at_runtime"`
}

ConfigEntry reports an exact stored entry rather than an inherited/default value. Configuration may contain sensitive values; log selected fields only.

type ConfigOverride 

type ConfigOverride struct {
	// contains filtered or unexported fields
}

ConfigOverride restores one exact database entry. Copies share lifecycle state. Restore explicitly when the test condition should end; cluster termination disposes of the entire configuration database. A non-nil handle returned with an error must also be restored or the cluster terminated.

func (ConfigOverride) GoString 

func (change ConfigOverride) GoString() string

GoString formats the override like String.

func (*ConfigOverride) Restore 

func (change *ConfigOverride) Restore(ctx context.Context) error

Restore returns the exact entry to its prior value, or removes it if it was absent. It refuses a different current value to preserve outside edits. After an uncertain restore reply, retrying reconciles the native state first. Same value outside writes cannot be distinguished; external edits must not race.

func (ConfigOverride) String 

func (change ConfigOverride) String() string

String names the override's section and option without its values.

type ConfigSetting 

type ConfigSetting struct {
	Section, Mask, Name, Value string
}

ConfigSetting addresses one entry in the MON configuration database. Section is global, mon, mgr, osd, mds, client or a specific identity such as osd.0. Mask is optional: class:ssd, host:node-a or host:node-a/class:ssd. These stored settings have lower precedence than local files, argv and runtime overrides.

type Container 

type Container struct {
	testcontainers.Container
	// contains filtered or unexported fields
}

Container embeds the MON/control container and owns the MGR, OSDs, optional RGW/MDS services and network. Use Terminate to clean up the entire cluster. Generic Run options customize the MON; module options configure the cluster. Use WithHostNetwork to change cluster networking, rather than changing only the MON's network mode.

func Run 

func Run(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*Container, error)

Run creates configurable MON, MGR and OSD containers for a disposable cluster. img must satisfy the control role: MON/MGR, CLI, Python clients and mirrors. Other roles use img unless overridden by WithOSDImage, WithMDSImage or rgw.WithImage. Images contain runtime tools; this module supplies cluster setup. A non-nil Container returned with an error must still be terminated.

Example 
package main

import (
	"context"
	"log"

	"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
	"github.com/testcontainers/testcontainers-go"
)

func main() {
	ctx := context.Background()
	cluster, err := ceph.Run(ctx, ceph.DefaultImage, ceph.WithOSDCount(2))
	defer func() {
		if err := testcontainers.TerminateContainer(cluster); err != nil {
			log.Printf("failed to terminate container: %s", err)
		}
	}()
	if err != nil {
		panic(err)
	}
	osd, err := cluster.AddOSD(ctx)
	if err != nil {
		panic(err)
	}
	if err := cluster.RemoveOSD(ctx, osd.ID); err != nil {
		panic(err)
	}
}
Example (Services)

Options from several service packages start one cluster that serves RBD, CephFS and S3 clients together. 

package main

import (
	"context"
	"log"

	"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
	"github.com/JSYoo5B/ceph-testcontainers-go/cephfs"
	"github.com/JSYoo5B/ceph-testcontainers-go/rbd"
	"github.com/JSYoo5B/ceph-testcontainers-go/rgw"
	"github.com/testcontainers/testcontainers-go"
)

func main() {
	ctx := context.Background()
	cluster, err := ceph.Run(ctx, ceph.DefaultImage,
		ceph.WithOSDCount(3),
		rbd.WithPools(ceph.PoolConfig{Name: "volumes"}),
		cephfs.WithFilesystems(cephfs.Config{Name: "shared"}),
		rgw.WithGateways(rgw.Config{Name: "s3"}),
	)
	defer func() {
		if err := testcontainers.TerminateContainer(cluster); err != nil {
			log.Printf("failed to terminate container: %s", err)
		}
	}()
	if err != nil {
		panic(err)
	}
	_ = cephfs.Filesystems(cluster)
	_ = rgw.Gateways(cluster)
}

func (*Container) AddManager 

func (c *Container) AddManager(ctx context.Context, name string) (*ManagerContainer, error)

AddManager creates a named active/standby candidate and waits for its mgrmap registration. Existing daemon identities are rejected; it does not force active ownership away from an existing MGR.

func (*Container) AddMonitor 

func (c *Container) AddMonitor(ctx context.Context, name string) (*MonitorContainer, error)

AddMonitor joins a new named MON from the live quorum's map and shared key. Existing daemon data is retained; a partial new member remains owned on error.

func (*Container) AddOSD 

func (c *Container) AddOSD(ctx context.Context) (*OSDContainer, error)

AddOSD registers, formats and starts a new OSD, then waits for it to be up/in. Topology operations are serialized. An OSD returned with an error is tracked for cleanup and may be inspected; no data is silently purged on failure.

func (*Container) AddOSDWithConfig 

func (c *Container) AddOSDWithConfig(ctx context.Context, config OSDConfig) (*OSDContainer, error)

AddOSDWithConfig starts an OSD in an explicit logical CRUSH location. Omitted host names resolve to osd-ID; logical hosts/racks are simulated placement domains on the same Docker engine, not physical failure domains.

func (*Container) AddressFamily added in v0.2.0 

func (c *Container) AddressFamily() AddressFamily

AddressFamily returns the family that the cluster was created with. A nil Container reports AddressIPv4.

func (*Container) AdminSocket added in v0.2.0 

func (c *Container) AdminSocket(ctx context.Context, daemon string, args ...string) ([]byte, error)

AdminSocket runs a command on the admin socket of one MON, MGR or OSD that this cluster owns and returns the command's output. daemon is the Ceph daemon name, such as "mon.a", "mgr.b" or "osd.0", and args are the command words, such as "status", "version" or "config", "get", "osd_heartbeat_grace"; "help" lists the daemon's commands.

The command does not use the Messenger or Cephx: it reaches the daemon through its local socket, so it works while the daemon cannot reach a monitor and reports what the daemon itself has applied, for example the newest OSDMap epoch in an OSD's "status". It runs in the control container, which shares the daemons' run directory, so it also works with OSD images that have no Ceph CLI. The daemon must be running; a stopped or removed daemon returns an error. A paused daemon accepts the connection but does not answer, so the call waits for ctx or the cluster's startup timeout, which also bounds every call. When the call ends early, AdminSocket stops its CLI in the control container before it returns, including one that Docker has not started yet. A nonzero exit returns the output with an error.

func (*Container) BlocklistEntries 

func (c *Container) BlocklistEntries(ctx context.Context) ([]BlocklistEntry, error)

BlocklistEntries lists exact entries and CIDR ranges without mutating them.

func (*Container) Ceph 

func (c *Container) Ceph(ctx context.Context, args ...string) ([]byte, error)

Ceph runs the Ceph CLI in the control container. Arguments are passed directly, without shell interpolation. Use --format json for machine-readable responses.

func (*Container) CephVersion 

func (c *Container) CephVersion() string

CephVersion reports the release of the image that runs the first MON and the fixture's Ceph CLI, such as "20.2.4". Run records it once from "ceph --version" after the MON starts and fails if the output cannot be parsed. Daemons started from other role images may run another release. The fixture uses it to choose commands that differ between releases, and tests can use it to choose assertions for features that a release lacks.

func (*Container) ClientCapabilities 

func (c *Container) ClientCapabilities(ctx context.Context, client *ClientConfig) (ClientCaps, error)

ClientCapabilities returns the current caps of a confirmed, owned identity. It verifies the native key without returning auth database secrets.

func (*Container) ClusterNetworkName 

func (c *Container) ClusterNetworkName() string

ClusterNetworkName returns the replication bridge. With a single bridge it returns NetworkName; in host mode both traffic planes share "host".

func (*Container) CollectCrashMetadata added in v0.2.0 

func (c *Container) CollectCrashMetadata(ctx context.Context, config CrashMetadataConfig) (*CrashMetadataReport, error)

CollectCrashMetadata reads the Ceph crash metadata files in CrashDir of the cluster's MON, MGR, OSD, MDS and RGW containers, and of config.AdditionalContainers, through Docker's archive API. It works on stopped containers and on a partially bootstrapped or terminated cluster, and starts, stops or reconfigures nothing. It reads only regular files named meta in a crash report directory, keeps their known fields and redacts them like CollectDiagnostics; it skips crash logs and every other file, and rejects links, unexpected paths and files that are not JSON objects. Each container's outcome, rejected files and errors stay in the report, which is returned also with an error for failed or incomplete reads. Crash metadata can still hold host names and paths; review a report before sharing it.

func (*Container) CollectDiagnostics 

func (c *Container) CollectDiagnostics(ctx context.Context, config DiagnosticsConfig) (*DiagnosticsReport, error)

CollectDiagnostics collects all tracked MON/MGR/OSD/MDS/RGW/control handles, container state, network addresses, bounded log tails and fixed Ceph queries. It supports partial bootstrap and stopped/removed containers. Individual failures return both a usable report and an aggregate error; truncation alone does not return an error. A canceled caller needs a fresh context to collect. No daemon is started, stopped or reconfigured, and no file, environment, keyring, auth database, mount source or arbitrary command is collected.

func (*Container) Configuration 

func (c *Container) Configuration(ctx context.Context) ([]ConfigEntry, error)

Configuration lists exact central database entries, including masked entries. Use native config show/daemon config get to inspect a running daemon's effective value; this method does not claim a stored setting has overridden local config.

func (*Container) ConnectionConfig 

func (c *Container) ConnectionConfig() ([]byte, []byte, error)

ConnectionConfig returns independent copies of ceph.conf and the ephemeral admin keyring for a native client. In host mode the MON ports are final.

func (*Container) ConnectionConfigContext 

func (c *Container) ConnectionConfigContext(ctx context.Context) ([]byte, []byte, error)

ConnectionConfigContext returns independent bootstrap copies, waiting for topology and the configuration cache only while ctx remains active.

func (*Container) ControlContainer 

func (c *Container) ControlContainer() testcontainers.Container

ControlContainer returns the stable CLI container for multi-monitor clusters, or the primary MON for the default single-monitor fixture. The cluster owns it.

func (*Container) ControlContainerContext 

func (c *Container) ControlContainerContext(ctx context.Context) (testcontainers.Container, error)

ControlContainerContext returns the stable CLI handle while bounding a concurrent control-plane creation or termination publication wait by ctx. The cluster owns the handle; this snapshot does not promise it remains live.

func (*Container) ControlImage 

func (c *Container) ControlImage() string

ControlImage returns the image selected for MON/MGR and auxiliary CLI/client containers. It is fixed by Run and remains available after termination. Multicluster setup clients use this image instead of a daemon-only RGW image.

func (*Container) CreateClient 

func (c *Container) CreateClient(ctx context.Context, name string, caps ClientCaps, opts ...ClientOption) (*ClientConfig, error)

CreateClient creates a fresh client with exactly the supplied capabilities. Existing identities are rejected before mutation; their keys and caps are never updated. A non-nil result with an error identifies an attempted or partial creation; inspect auth state or terminate the cluster rather than retrying with broader caps. DeleteClient can revoke a confirmed successful creation. The key has the cluster's AuthConfig.KeyType, the release default without one, or the type that WithClientKeyType selects.

func (*Container) CreatePool 

func (c *Container) CreatePool(ctx context.Context, config PoolConfig) (*Pool, error)

CreatePool creates a new pool with explicit placement and replication settings. It validates the entire config and requires enough eligible owned domains before sending any Ceph command. Existing pools, profiles and rules are rejected rather than changed. A non-nil Pool returned with an error identifies resources whose creation was attempted; inspect them or terminate the disposable cluster. It never deletes data after a partial command failure. Use WaitForClean before application I/O. For RBD, create images in a replicated metadata pool and pass the EC pool as data_pool in the native client (or --data-pool to the RBD CLI).

func (*Container) DaemonAddresses added in v0.2.0 

func (c *Container) DaemonAddresses(ctx context.Context) (DaemonAddresses, error)

DaemonAddresses reads the names and advertised addresses of the cluster's MONs, MGRs and OSDs from the native maps. They are the addresses a client learns from the cluster, which can differ from the addresses a container has: a host-network cluster advertises its host address, and an OSD that is down keeps its last addresses.

func (*Container) DeepScrubPG 

func (c *Container) DeepScrubPG(ctx context.Context, pgid string) error

DeepScrubPG requests a native deep scrub of one PG and waits until the PG reports a deep-scrub stamp newer than before the request. Inconsistencies are reported, not repaired. The noscrub and nodeep-scrub flags or an unavailable primary can keep the wait from finishing before its deadline.

func (*Container) DeleteClient 

func (c *Container) DeleteClient(ctx context.Context, client *ClientConfig) error

DeleteClient revokes an identity created by this cluster without deleting data. It checks for an externally replaced key before deleting; external auth edits must not race this operation. A repeat call after success is a no-op; existing sessions may retain issued tickets. The removal is fenced like the import of ReplaceClientKey: a failed call reports what the MONs hold once settled, and an uncertain one is resolved by the next operation on the identity.

func (*Container) FullRatios 

func (c *Container) FullRatios(ctx context.Context) (FullRatioSnapshot, error)

FullRatios reads native thresholds through one captured control CLI handle, checking the original bootstrap FSID before and after the map query. Native zero, equal and misordered thresholds remain observable. It makes no writes, retries or background workers and returns a zero snapshot on any failure. These separate reads are not atomic; external configuration writers must not race the observation. Poll HealthDetails for eventual OSD fullness reports.

func (*Container) HasSeparateClusterNetwork 

func (c *Container) HasSeparateClusterNetwork() bool

HasSeparateClusterNetwork reports whether OSDs have a dedicated second bridge.

func (*Container) HealthDetails 

func (c *Container) HealthDetails(ctx context.Context) (HealthSnapshot, error)

HealthDetails reads the original cluster's health through one captured control CLI handle, checking the bootstrap FSID before and after the query. It works before storage or MGR readiness, performs no writes or retries and returns a zero snapshot on every failure, including post-query cancellation. Native command output is omitted from errors because it may contain secrets. External native/configuration writers must not race this observation: the three reads are not an atomic transaction. Poll with a caller deadline when asserting eventual health after changing a test topology.

func (*Container) InjectObjectDataError 

func (c *Container) InjectObjectDataError(ctx context.Context, pool, object string, osdID int) error

InjectObjectDataError makes one replica of an object on an owned OSD return a read error to scrub and recovery, through the native injectdataerr debug command. The OSD must be in the object's current acting set of a replicated pool. The command needs bluestore_debug_inject_read_err, which this method enables on that daemon only through its runtime configuration; it is not stored centrally and lasts until the OSD restarts. The setting has no effect on objects without an injected error.

The injection stays in OSD memory until the replica is rewritten, such as by RepairPG, or the OSD restarts. Client reads served by a healthy primary are unaffected. Use DeepScrubPG to report the damage and PGInconsistencies, HealthDetails (OSD_SCRUB_ERRORS, PG_DAMAGED) or PoolPGs to observe it. It needs BlueStore OSDs and refuses a cluster that uses OSDMemStore.

func (*Container) InterruptNetwork 

func (c *Container) InterruptNetwork(ctx context.Context, ctr testcontainers.Container, plane NetworkPlane) (*NetworkInterruption, error)

InterruptNetwork disconnects a container from one of this cluster's bridges. The container may be an owned daemon or a caller-owned WithClient container. With a single bridge, both planes select that same endpoint. Host mode is rejected because Docker cannot disconnect a host-network endpoint. The fixture retains the handle and attempts restoration during Terminate. Restore explicitly before destroying caller-owned containers or the cluster.

func (*Container) MGRModules 

func (c *Container) MGRModules(ctx context.Context) ([]MGRModuleState, error)

MGRModules lists configured and available modules, sorted by name. Enabled and CanRun do not prove command readiness; WaitMGRModuleReady probes selected built-in modules, and callers must probe their intended commands for others.

func (*Container) ManagerContainer 

func (c *Container) ManagerContainer() testcontainers.Container

ManagerContainer returns the initial MGR a for compatibility and inspection. It may be nil after a is removed while another MGR remains active. Use Managers and ManagerStatus for the current candidates and active identity. The cluster owns its cleanup.

func (*Container) ManagerServices 

func (c *Container) ManagerServices(ctx context.Context) (map[string]string, error)

ManagerServices returns the service URLs, keyed by module name such as prometheus or dashboard, that the active manager advertises in the MGR map. A module appears only after it has started its listener, and a failover replaces the URLs with the new active manager's. The URLs carry the address the module bound: a container address reachable from WithClient containers in bridge mode, or the configured address in host mode. They are not mapped to Docker host ports. An empty map means no module advertises a service.

func (*Container) ManagerStatus 

func (c *Container) ManagerStatus(ctx context.Context) (ManagerStatus, error)

func (*Container) Managers 

func (c *Container) Managers() []*ManagerContainer

Managers returns active/standby candidates owned by this fixture.

func (*Container) ManagersContext 

func (c *Container) ManagersContext(ctx context.Context) ([]*ManagerContainer, error)

ManagersContext returns sorted owned candidate descriptors. The slice is independent; descriptors and their containers retain fixture ownership.

func (*Container) MessengerMode 

func (c *Container) MessengerMode() MessengerMode

MessengerMode returns the selected bootstrap policy. It does not inspect effective native settings or prove the mode negotiated by a live connection. A nil Container reports MessengerDefault.

func (*Container) MonitorBootstrapAddresses 

func (c *Container) MonitorBootstrapAddresses(ctx context.Context) (string, error)

MonitorBootstrapAddresses returns the current majority monmap's versioned address vectors, suitable for the Ceph mon_host setting. The live cluster FSID must match this fixture's original bootstrap configuration. This is a bounded native observation, not a network attachment or membership mutation.

func (*Container) Monitors 

func (c *Container) Monitors() []*MonitorContainer

Monitors returns membership owned by this fixture, sorted by daemon name.

func (*Container) NetworkName 

func (c *Container) NetworkName() string

NetworkName returns the cluster's bridge name, or "host" in host mode.

func (*Container) OSDFlags 

func (c *Container) OSDFlags(ctx context.Context) ([]string, error)

OSDFlags lists exact global OSDMap flag tokens, including native compatibility flags. Per-OSD, CRUSH-node or device-class flags are separate native state.

func (*Container) OSDStates 

func (c *Container) OSDStates(ctx context.Context) ([]OSDState, error)

OSDStates returns owned OSD states sorted by native ID. It refuses an owned ID that is missing, purged or now belongs to a different native UUID.

func (*Container) OSDs 

func (c *Container) OSDs() []*OSDContainer

OSDs returns a snapshot of owned OSD containers sorted by ID.

func (*Container) ObjectWatchers 

func (c *Container) ObjectWatchers(ctx context.Context, pool, namespace, object string) ([]ObjectWatcher, error)

ObjectWatchers lists the clients watching one object, including clients outside this fixture. Namespace is empty for the default namespace. It reads native state only and grants no ownership.

func (*Container) PGInconsistencies 

func (c *Container) PGInconsistencies(ctx context.Context, pgid string) ([]InconsistentObject, error)

PGInconsistencies lists the objects that the last deep scrub of a PG found inconsistent. An empty result means that scrub found none; it is not a new scrub. A PG whose results are no longer available returns an error.

func (*Container) PauseContainer 

func (c *Container) PauseContainer(ctx context.Context, ctr testcontainers.Container) (*ContainerPause, error)

PauseContainer freezes every process of an owned daemon or a caller-owned WithClient container, as with docker pause. Unlike Stop, the process keeps its memory, sockets and Ceph identity, and peers see silence rather than a closed connection: a frozen OSD stops answering client operations and heartbeats until its peers report it down after the heartbeat grace. Hold nodown with TemporaryOSDFlag to keep it up and make client operations wait for their own timeouts or be reported as slow. Host networking is supported.

The container that runs this cluster's control CLI is refused because every fixture command would block on it. A container that is already paused is refused rather than adopted. The cluster retains the handle and resumes it during Terminate; resume explicitly before removing a caller-owned container.

func (*Container) PoolPGs 

func (c *Container) PoolPGs(ctx context.Context, name string) (PoolPGSnapshot, error)

PoolPGs observes one existing pool through one captured control CLI handle. It checks the original bootstrap FSID and the exact pool name/ID before and after pg ls-by-pool, using five CLI reads and no per-PG mapping fanout. It works before any OSD is started when a native pool has been prepared through Ceph. Successful empty/unready/unknown reports are retained as observations; no clean, health, owned-OSD membership or expected PG-count predicate applies.

All failures, including post-query cancellation, return a zero snapshot and static errors that omit native output. There are no writes, retries or workers. External native/configuration writers must not race these separate reads. Compare original fixture identities and data when asserting movement; these reported PGs are not current mappings or read-after-write evidence.

func (*Container) PoolStatus 

func (c *Container) PoolStatus(ctx context.Context, name string) (PoolState, error)

PoolStatus looks up a pool by its exact native name.

func (*Container) PoolUsage 

func (c *Container) PoolUsage(ctx context.Context, name string) (PoolUsageSnapshot, error)

PoolUsage reads ceph df detail through the fixture's control CLI. It checks the original bootstrap FSID and exact native pool ID/name before and after the observation. A pool without a statistics row is unavailable, not empty. All failures, including cancellation after a query, return a zero snapshot. This method makes no writes, retries or background workers. External pool or control configuration writers must not race it; the native reads are not an atomic transaction. Poll with a caller deadline for eventual assertions.

func (*Container) Pools 

func (c *Container) Pools(ctx context.Context) ([]PoolState, error)

Pools lists native pool policies. Creation descriptors returned by CreatePool describe requested settings; this method reports the current cluster state.

func (*Container) PublicAddress 

func (c *Container) PublicAddress() string

PublicAddress returns the advertised host-mode IPv4 address; empty in bridge mode.

func (*Container) QuorumStatus 

func (c *Container) QuorumStatus(ctx context.Context) (QuorumStatus, error)

func (*Container) RedactSecrets added in v0.2.0 

func (c *Container) RedactSecrets(ctx context.Context, data []byte, values ...string) ([]byte, error)

RedactSecrets returns a copy of data in which the secrets that the cluster knows, and values, are replaced by [REDACTED], as in CollectDiagnostics: keyring keys, service credentials, and the private keys, secret assignments and bearer tokens that it recognizes. It is meant for output that a test logs or reports; an oracle compares the raw bytes. Redaction is best effort.

func (*Container) RefreshClientMonitorConfig 

func (c *Container) RefreshClientMonitorConfig(ctx context.Context, client testcontainers.Container) error

RefreshClientMonitorConfig updates only the explicit client's global mon_host entry in /etc/ceph/ceph.conf. The client's global FSID must match this fixture and its live quorum. Docker archive copies also work with stopped containers; this method never starts, stops, terminates or adopts the client's lifetime. Other settings and keyring files are retained. It does not update mirror peer attributes, network attachments, or private-section mon_host overrides.

Successful writes remain on partial failure and may be retried with a fresh context. External configuration writers must not race this operation: Docker has no file compare-and-swap. Callers own clients and their cleanup.

func (*Container) RefreshMonitorConfig 

func (c *Container) RefreshMonitorConfig(ctx context.Context) error

RefreshMonitorConfig copies the live quorum's bootstrap addresses into the global mon_host entry of every owned daemon's ceph.conf, including stopped daemons, without starting or restarting them. Other entries and comments are retained. Caller-owned clients and multicluster daemon configs are not copied.

AddMonitor and RemoveMonitor already refresh these files. Retry this method after an AddMonitor that joined native membership but could not copy every config; retry RemoveMonitor itself after a partial removal. Successful copies and the future-client template are retained on partial failure. Repeating a refresh reads each current file and skips copies that are already up to date. External config writers must not race this operation: Docker has no file CAS.

func (*Container) RemoveManager 

func (c *Container) RemoveManager(ctx context.Context, name string) error

RemoveManager deletes an owned manager after another running, registered candidate is available. Add a replacement before removing the last candidate. On failure the handle remains owned for retry or cluster cleanup. Do not edit these manager identities externally while a topology operation is in progress.

func (*Container) RemoveMonitor 

func (c *Container) RemoveMonitor(ctx context.Context, name string) error

RemoveMonitor preserves a working quorum; add a replacement first when a stop would leave fewer than a majority of the current map running.

func (*Container) RemoveOSD 

func (c *Container) RemoveOSD(ctx context.Context, id int) error

RemoveOSD drains an owned OSD, waits for safe-to-destroy, stops it, waits for down, purges it from Ceph and removes its container. It refuses the last OSD. Every removal phase verifies the captured registration UUID. External OSD replacement must not race removal: Ceph has no UUID compare-and-swap purge. An uncertain purge response remains tracked; a retry reconciles fresh native absence only after this handle issued purge. Completed purge retries only Docker cleanup. On error retry with a fresh context or terminate the cluster.

func (*Container) RemovePool 

func (c *Container) RemovePool(ctx context.Context, name string) error

RemovePool deletes a pool created by CreatePool or WithPools, together with the CRUSH rule and erasure-code profile created for it, so a long-lived cluster can give each test its own pool without reaching PG limits. Pools that Ceph or a service created (.mgr, RGW zone pools, CephFS pools) and pools still used by a CephFS filesystem are refused. Every object in the pool is destroyed, including RBD images and namespaces; stop clients and mirroring that use it first. mon_allow_pool_delete is enabled in the central database only while the pool is removed and then restored; a local config file that sets it to false blocks removal. Ownership is recognized by the pool's tc-<name>-replicated or tc-<name>-ec CRUSH rule, so a pool that SetPoolPlacement moved to another rule is refused until it moves back; placement rules are not removed. A retry after an error finishes removing whichever owned resources remain. External edits to the pool must not race this call.

func (*Container) RepairPG 

func (c *Container) RepairPG(ctx context.Context, pgid string) error

RepairPG requests a native repair of one PG and waits until the PG reports a newer deep-scrub stamp and is no longer inconsistent. Repair rewrites bad replicas from an authoritative copy; it cannot recover an object without one, and the wait then ends with the PG still inconsistent.

func (*Container) ReplaceClientKey added in v0.2.0 

func (c *Container) ReplaceClientKey(ctx context.Context, client *ClientConfig, opts ...ClientOption) (*ClientConfig, error)

ReplaceClientKey gives a client identity that this cluster created a fresh Cephx key and keeps its name and its exact capabilities, in one auth import, so that the MONs never lack the identity. The identity needs a capability, as auth import refuses an entry without any. The new key has the type of the old one unless WithClientKeyType selects another. It returns a new descriptor for the new key, and the old descriptor goes stale: its ConnectionConfig still returns the old credentials, for tests that check their rejection, while every other operation on it fails. Containers that were attached with the old descriptor keep the old keyring.

A fresh connection with the old key fails at once. A client that authenticated before keeps its session and its tickets until it renews them within auth_mon_ticket_ttl (AuthConfig.MonTicketTTL), when the MONs reject the old key: Tentacle then drops the client's MON session, while Squid keeps an open one answering. The new key works for fresh MON and MGR clients at once.

The identity must still have the key of client; one changed outside the module is refused. External auth changes must not race this call. The import is fenced like the MON commands of RotateDaemonKey: when it fails, the call settles the MONs and reads whether it applied. When that stays uncertain, the call returns a descriptor of the attempted key that stays unusable, with the error, and a later call with client, or any other operation on the identity, settles first: ReplaceClientKey then returns a usable descriptor of that key once the import turns out to have applied, or replaces the key anew once it turns out not to have. A returned descriptor never changes afterwards.

func (*Container) RotateDaemonKey added in v0.2.0 

func (c *Container) RotateDaemonKey(ctx context.Context, daemon string) (err error)

RotateDaemonKey replaces the Cephx key of a running MGR or OSD that the cluster owns, as cephadm's daemon key rotation does, without a restart:

  1. auth get-or-create-pending creates the new key on the MONs, which then accept both keys.
  2. The daemon's keyring files in its container get the new key, so that a later restart authenticates with it.
  3. ceph tell <daemon> rotate-key hands the key to the running daemon.
  4. auth commit-pending makes it the entity's only key.

daemon is "mgr.<name>" or "osd.<id>". MONs share the mon. key and are refused.

The MONs accept both keys until step 4, and they also commit the pending key on their own when the daemon authenticates with it, after a restart or a handoff. A failure before step 4 therefore keeps both keys valid, and the next call finishes the rotation with the same key; a failed step 4 is reported with the outcome that the MONs show once settled. A call returns only once none of its commands can still change the keys or the keyring files, or after recording such commands for the daemon: the next call stops them, waits for them and settles the MONs for them before it changes anything, and fails while it cannot. After a handoff that may still reach the daemon, a later call finishes the rotation with the same key and generates another key only once the daemon's container has restarted.

func (*Container) RunNativeCommand added in v0.2.0 

func (c *Container) RunNativeCommand(ctx context.Context, target testcontainers.Container, command NativeCommand) (NativeCommandResult, error)

RunNativeCommand runs one native command in target under the supervisor that the module's own native queries use. target is the control container, a daemon or service container of this cluster, or a container of the caller attached to the cluster's network (the host network for a host-network cluster); the call never adopts, stops or removes the caller's container. target must be running, not paused, and have python3, which runs the supervisor. A malformed command or an ended ctx fails before anything runs.

The supervisor starts the command in its own process group, bounds each output stream inside the container and kills the group at the deadline, Timeout shortened to ctx's deadline. A command that times out or exceeds an output limit returns its result, marked by TimedOut or the Truncated flags, with an error that wraps ErrNativeCommandIncomplete. A descendant that starts its own session leaves the group and is not stopped.

When ctx ends first, the call stops the command inside the container before it returns, also when Docker had not started it yet, within its own NativeCommandStopTimeout. When that cannot be confirmed, for example because the container is paused or a signal killed the supervisor, the error is a *NativeCommandUnstoppedError, whose Stop retries. Terminate cancels running commands and stops those not confirmed. Stopping a command does not undo what it already did, such as a MON command it had sent, and nothing is retried. Errors name the command by Label; a failure of Docker or of target's implementation appears only by its kind, and errors.Unwrap returns the original error, whose text may quote anything. RedactSecrets redacts output that a test logs.

With Input, the supervisor runs in an exec of the Docker engine with its standard input attached, and passes what Input writes to the command; see NativeInput. The call stops taking writes when the command ends, and returns only after its input and output work ended. Its deadline and cancellation stop the command as without Input, and a write in progress then fails. A malformed limit or an Input that served another command fails before anything runs or any input is taken.

func (*Container) ServiceContainers 

func (c *Container) ServiceContainers() []testcontainers.Container

ServiceContainers returns owned optional services (such as RGW or MDS), sorted by service name. The cluster owns their cleanup.

func (*Container) SetOSDIn 

func (c *Container) SetOSDIn(ctx context.Context, id int, in bool) error

SetOSDIn changes native OSDMap membership of an existing owned OSD. It does not stop, purge or change CRUSH capacity weights. Readback verifies the owned UUID and desired state, since native in/out can succeed for nonexistent IDs. WaitForPGClean can wait for recovery while an OSD remains out. Ceph preserves a nonzero map reweight across out/in. External replacement must not race this method; Ceph has no compare-and-swap operation for OSD membership.

func (*Container) SetPoolPGCount 

func (c *Container) SetPoolPGCount(ctx context.Context, name string, pgNum int) error

SetPoolPGCount requests a new PG count for an existing pool with autoscaling off or warn. It records both native pg_num and pgp_num targets; the active manager then splits or merges PGs in steps, and merges wait for clean PGs. Success means the targets were accepted, not that PGs have changed. Use WaitForPoolPGCount for the count and WaitForClean for the resulting movement. Without an active manager the targets remain pending. The native pool ID is checked before each mutation; on partial failure inspect PoolStatus. Native per-OSD PG limits can still reject the request.

func (*Container) SetPoolPlacement 

func (c *Container) SetPoolPlacement(ctx context.Context, name string, placement PoolPlacement) error

SetPoolPlacement moves a replicated pool to a simple osd/host/rack CRUSH rule. The size and min_size are unchanged and the new placement needs enough owned eligible domains in the native CRUSH map. It reuses the pool's CreatePool rule or an earlier rule from this method when its native steps match; otherwise it creates tc-<pool>-placement-<hash>. Rules are not deleted, so a pool can move back. Success means the pool references the rule; data moves asynchronously. Until PG reports reflect the new map, WaitForClean can pass on the previous placement, so poll PoolPGs for the expected up/acting sets. Holding both nobackfill and norecover with TemporaryOSDFlag keeps the move incomplete; nobackfill alone does not stop log-based recovery of small PGs. EC rules are bound to their profile and are refused. External pool/CRUSH edits must not race this call.

func (*Container) SetPoolQuota 

func (c *Container) SetPoolQuota(ctx context.Context, name string, quota PoolQuota) error

SetPoolQuota replaces both limits, including zero values that remove a limit. Commands are sequential and may partially succeed; inspect PoolStatus after an error. Existing data is never removed. External pool edits must not race this operation; native pool IDs are checked before each mutation.

func (*Container) SetPoolReplication 

func (c *Container) SetPoolReplication(ctx context.Context, name string, replicas, minSize int) error

SetPoolReplication changes a replicated pool's size/min_size without changing data, PG count or placement. It requires enough owned placement domains in the native CRUSH rule. EC shard counts cannot be changed by this method. WaitForClean waits for resulting recovery. On partial failure, inspect PoolStatus; no rollback or data deletion is attempted. External pool/CRUSH edits must not race this call.

func (*Container) Status 

func (c *Container) Status(ctx context.Context) (Status, error)

Status queries the monitor's JSON API through the container-local CLI.

func (*Container) TemporaryBlocklist 

func (c *Container) TemporaryBlocklist(ctx context.Context, address string, duration time.Duration) (*BlocklistOverride, error)

TemporaryBlocklist fences a specific native session for a positive duration of at least one second. Use librados GetAddrs (or its equivalent) after connect. A v1/v2 address vector is accepted only when every entry denotes the same IP, port and nonce; otherwise select one address explicitly. Protocol types become native ANY entries. Bare IPs, CIDRs, nonce zero and existing entries are refused.

The handle is retained on uncertain command/readback failure. Restore checks its captured expiration before removal, preserving outside renewal. An unknown readback cannot authorize removal of a present entry; inspect native state or terminate the disposable cluster. Natural expiry is reconciled as absence.

func (*Container) TemporaryClientRevocation added in v0.2.0 

func (c *Container) TemporaryClientRevocation(ctx context.Context, client *ClientConfig) (*ClientRevocation, error)

TemporaryClientRevocation removes a client identity that this cluster created from the MONs until Restore puts back exactly its key and capabilities, so that a test can observe its rejection and then the recovery of the same credentials. The identity must still have the key of client; one changed outside the module is refused. While it is revoked, the descriptor works only for ConnectionConfig, whose credentials stay valid for after Restore; WithClientIdentity, ClientCapabilities, UpdateClientCaps and ReplaceClientKey fail, CreateClient refuses its name, and DeleteClient makes the removal permanent.

A fresh connection with the client's key fails at once. A client that authenticated before keeps its sessions until it renews its tickets, within auth_mon_ticket_ttl (AuthConfig.MonTicketTTL), when the MONs reject the renewal; it keeps the MGR and OSD sessions it already holds. Tentacle also drops its MON session, and a librados client then does not authenticate again on its own, while Squid keeps an open MON session answering. A new connection with the same credentials works once Restore returns.

The removal and the restoration are fenced like the import of ReplaceClientKey. A failed call settles the MONs and reports what they hold. When that stays uncertain, TemporaryClientRevocation returns the revocation with the error, and Restore settles first and then restores. The cluster needs no Restore before Terminate.

func (*Container) TemporaryConfig 

func (c *Container) TemporaryConfig(ctx context.Context, setting ConfigSetting) (*ConfigOverride, error)

TemporaryConfig sets one central database entry and records its native normalized value. It preserves absence separately from inherited defaults, and rejects overlapping handles for the same section/mask/name. On command or readback failure, the handle remains tracked for Restore. Never race external writes to this key with application/restoration; Ceph has no config CAS. MessengerV2Secure reserves its six ms_*_mode options and ms_bind_msgr1/2 in every section and mask: central overrides cannot change the fixed local bootstrap policy. MessengerV2Only reserves ms_bind_msgr1/2 the same way and leaves the mode options free. Select a MessengerMode when creating a new cluster instead.

Example 
package main

import (
	"context"
	"log"

	"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
	"github.com/testcontainers/testcontainers-go"
)

func main() {
	ctx := context.Background()
	cluster, err := ceph.Run(ctx, ceph.DefaultImage)
	defer func() {
		if err := testcontainers.TerminateContainer(cluster); err != nil {
			log.Printf("failed to terminate container: %s", err)
		}
	}()
	if err != nil {
		panic(err)
	}
	change, err := cluster.TemporaryConfig(ctx, ceph.ConfigSetting{
		Section: "osd", Name: "osd_max_backfills", Value: "1",
	})
	if change != nil {
		// A failed apply can still have persisted; retain its restoration handle.
		defer func() {
			if err := change.Restore(ctx); err != nil {
				panic(err)
			}
		}()
	}
	if err != nil {
		panic(err)
	}
	// Exercise the native client, then Restore removes an originally absent
	// entry instead of pinning the daemon's inherited/default value.
}

func (*Container) TemporaryFullRatios 

func (c *Container) TemporaryFullRatios(ctx context.Context, requested FullRatios) (*FullRatiosOverride, error)

TemporaryFullRatios applies finite thresholds satisfying 0 < NearFull < BackfillFull < Full <= 1 after native float32 normalization. The original map must also have this ordering. Overlapping handles are refused, including no-op leases. Each intermediate commit preserves ordering. Success verifies map values, not OSD reports or client write availability. External writers must not race apply/restore: Ceph has no threshold CAS and writes equal to owned values cannot be distinguished from this fixture.

func (*Container) TemporaryMGRModule 

func (c *Container) TemporaryMGRModule(ctx context.Context, name string, enabled bool) (*MGRModuleOverride, error)

TemporaryMGRModule changes configured membership without forcing unavailable dependencies. It rejects missing/cannot-run metadata on enable, overlapping handles, always-on disable, and disabling volumes or mirroring while relevant filesystems/mirror policies exist. An enabled always-on module accepts only a no-op true lease; a force-disabled always-on module cannot be temporarily enabled because restoring it would require force-disable. This method verifies membership, not command readiness. External edits must not race apply/restore; Ceph has no module membership CAS or per-module generation.

func (*Container) TemporaryMessengerPolicy added in v0.2.0 

func (c *Container) TemporaryMessengerPolicy(ctx context.Context, policy MessengerPolicy) (*MessengerPolicyOverride, error)

TemporaryMessengerPolicy changes the connection modes that the running daemons accept from clients. Each non-nil field becomes a TemporaryConfig override of the policy's option; nil fields stay as they are. The daemons apply the change to new connections without a restart, and established connections keep their mode. A non-nil result returned with an error holds the overrides that were made and must be restored. A policy that selects no valid mode, and any policy with MessengerV2Secure, is refused with ErrInvalidRequest before any change.

func (*Container) TemporaryNetworkDelay added in v0.2.0 

func (c *Container) TemporaryNetworkDelay(ctx context.Context, target testcontainers.Container, delay NetworkDelay) (*NetworkDelayLease, error)

TemporaryNetworkDelay delays every packet that target sends through its endpoint on the plane's bridge, below Ceph, with the kernel's netem qdisc. It is an opt-in extension: delay.HelperImage names the netem helper, which runs in the target's network namespace with NET_ADMIN, while the target keeps its own image and capabilities. The target is an owned daemon or a caller-owned container attached to that bridge; host-network clusters, stopped or paused targets and targets that share another container's network namespace are refused.

Before it changes anything, it checks the helper image against the netem contract, NET_ADMIN and the kernel's netem in a separate helper without a network, and returns ErrNetemImageUnsupported, ErrNetemPermissionDenied or ErrNetemKernelUnsupported. It refuses an interface whose root qdisc is not the kernel's default and a target that already has a delay on that bridge. A missing HelperImage is ErrImageCapability, the refusals are ErrInvalidRequest, and a tc command that fails otherwise is a *NativeFailure.

The delay lasts until Restore, or until the target stops or leaves the bridge, which removes its interface. Terminate restores delays that are still held; restore a caller-owned target's delay before removing it.

func (*Container) TemporaryOSDFlag 

func (c *Container) TemporaryOSDFlag(ctx context.Context, flag string, enabled bool) (*OSDFlagOverride, error)

TemporaryOSDFlag changes one reversible global flag while remembering its prior state. Supported flags are noup, nodown, noout, noin, nobackfill, norebalance, norecover, noscrub, nodeep-scrub, nosnaptrim and noautoscale. Composite pause and irreversible compatibility flags are excluded. Overlapping handles for a flag are refused. External edits to the same flag must not race apply/restore: a boolean flag has no generation and Ceph provides no CAS.

func (*Container) Terminate 

func (c *Container) Terminate(ctx context.Context, opts ...testcontainers.TerminateOption) error

Terminate removes all owned daemons before removing the isolated network. It also works on a partially initialized cluster returned by Run. The whole cluster is discarded, so each container gets terminationStopTimeout instead of Docker's 10 seconds to stop before it is killed; a StopTimeout option passed by the caller still takes precedence.

func (*Container) UpdateClientCaps 

func (c *Container) UpdateClientCaps(ctx context.Context, client *ClientConfig, caps ClientCaps) error

UpdateClientCaps replaces all capabilities while retaining the identity/key. Omitted services lose their previous capabilities. External auth edits must not race this call. Already issued session tickets may retain previous caps; verify permission changes using a fresh client connection. The change is fenced like the import of ReplaceClientKey: a failed call reports what the MONs hold once settled, and an uncertain one is resolved by the next operation on the identity.

func (*Container) UsesHostNetwork 

func (c *Container) UsesHostNetwork() bool

UsesHostNetwork reports whether the cluster shares Docker's host network.

func (*Container) WaitForClean 

func (c *Container) WaitForClean(ctx context.Context) error

WaitForClean waits until a manager is available, the native OSD count equals the owned OSD count with every OSD up and in, at least one PG exists and every PG is exactly active+clean. HEALTH_WARN alone does not block it. PG states with extra flags, such as active+clean+scrubbing, do not count. It never succeeds while an owned OSD is intentionally stopped or out; use WaitForPGClean then. The cluster's startup timeout also bounds the wait.

func (*Container) WaitForPGClean 

func (c *Container) WaitForPGClean(ctx context.Context) error

WaitForPGClean waits for every existing PG to be exactly active+clean. Unlike WaitForClean it allows intentionally out/down OSDs, and does not assert any OSD topology. At least one PG and an available manager are required. This is a recovery barrier, not a promise of cluster health or daemon availability.

func (*Container) WaitForPoolPGCount 

func (c *Container) WaitForPoolPGCount(ctx context.Context, name string, pgNum int) error

WaitForPoolPGCount waits until the native pool reports pgNum as its current, pending and target PG and PGP counts and the manager reports exactly pgNum PGs for it. Until then WaitForClean can pass on the previous PG set. It identifies the pool by the native ID first observed and fails if the name is later bound to another pool. The count does not imply clean PGs, completed backfill or data visibility; call WaitForClean afterwards or use PoolPGs. Cluster startup timeout applies.

func (*Container) WaitForQuorum 

func (c *Container) WaitForQuorum(ctx context.Context) error

WaitForQuorum waits for a majority of the current monmap, including fixtures with a stopped monitor. It does not require HEALTH_OK or every MON to run.

func (*Container) WaitMGRModuleReady 

func (c *Container) WaitMGRModuleReady(ctx context.Context, name string) error

WaitMGRModuleReady waits for enabled/can-run metadata and successful native read-only commands. Supported modules are rbd_support (task and snapshot schedule lists) and volumes (volume list). Other modules need an application specific command probe. Each poll releases the cluster lock; cancellation and termination stop the wait. Success does not claim worker tasks have completed.

func (*Container) WithClient 

func (c *Container) WithClient(opts ...ClientContainerOption) testcontainers.CustomizeRequestOption

WithClient connects another container to Ceph and copies its admin credentials. These privileged, ephemeral credentials are for trusted test containers only. Caller owns and terminates that container before terminating the cluster. The options customize the container's ceph.conf.

func (*Container) WithClientIdentity 

func (c *Container) WithClientIdentity(client *ClientConfig, opts ...ClientContainerOption) testcontainers.CustomizeRequestOption

WithClientIdentity attaches a container with only the selected client keyring. It rejects descriptors from another cluster, incomplete creation and revocation. Caller owns the attached container and must terminate it before the cluster. The options customize the container's ceph.conf.

type ContainerPause 

type ContainerPause struct {
	ContainerID string
	// contains filtered or unexported fields
}

ContainerPause owns one container frozen through Docker's cgroup freezer. Resume thaws it without restarting the process. A non-nil handle returned with an error must be resumed: a transport failure cannot prove that Docker did not freeze the container. Resume is retryable and idempotent.

func (*ContainerPause) Resume 

func (pause *ContainerPause) Resume(ctx context.Context) error

Resume thaws the container. A removed, stopped or already unpaused container needs no further action, so it completes the handle without starting it.

type CrashMetadataConfig added in v0.2.0 

type CrashMetadataConfig struct {
	Timeout              time.Duration
	OperationTimeout     time.Duration
	MaxArchiveBytes      int64
	MaxEntries           int
	MaxRecordBytes       int
	MaxRecords           int
	AdditionalContainers []DiagnosticsContainer
	RedactValues         []string
}

CrashMetadataConfig bounds CollectCrashMetadata. Zero fields select a one minute total timeout, 20 seconds per container, 16 MiB of archive and 4096 archive entries per container, 64 KiB per metadata file and 64 records per container. AdditionalContainers are caller-owned containers, such as mirror daemons, whose CrashDir is read too; collection never adopts them. RedactValues removes additional opaque secrets from the collected fields.

type CrashMetadataContainer added in v0.2.0 

type CrashMetadataContainer struct {
	Role        string `json:"role"`
	Name        string `json:"name"`
	ContainerID string `json:"container_id,omitempty"`
	ImageID     string `json:"image_id,omitempty"`
	CrashDir    string `json:"crash_dir"`
	State       string `json:"state,omitempty"`
	ExitCode    int    `json:"exit_code"`
	OOMKilled   bool   `json:"oom_killed,omitempty"`
	StartedAt   string `json:"started_at,omitempty"`
	FinishedAt  string `json:"finished_at,omitempty"`

	Outcome CrashOutcome  `json:"outcome"`
	Error   string        `json:"error,omitempty"`
	Records []CrashRecord `json:"records"`
	// Omitted lists the metadata files that were rejected or not read.
	Omitted []CrashOmission `json:"omitted,omitempty"`
	// IgnoredEntries counts the other files, such as crash logs, whose
	// content was skipped.
	IgnoredEntries int `json:"ignored_entries"`
	// ArchiveBytes counts the archive bytes read.
	ArchiveBytes int64 `json:"archive_bytes"`
}

CrashMetadataContainer is what one container's crash directory held. The state fields are Docker's view at collection, for correlation with the records.

type CrashMetadataReport added in v0.2.0 

type CrashMetadataReport struct {
	SchemaVersion int                      `json:"schema_version"`
	StartedAt     time.Time                `json:"started_at"`
	FinishedAt    time.Time                `json:"finished_at"`
	Closed        bool                     `json:"closed"`
	Complete      bool                     `json:"complete"`
	Containers    []CrashMetadataContainer `json:"containers"`
}

CrashMetadataReport is an independent, JSON-serializable report of the Ceph crash metadata in the cluster's daemon containers. Complete means that every container's crash directory was read in full and every metadata file in it was accepted, or that the directory is absent. It does not mean that every crash in the cluster's history was captured: a removed container takes its records along.

type CrashOmission added in v0.2.0 

type CrashOmission struct {
	Path   string `json:"path"`
	Reason string `json:"reason"`
}

CrashOmission names a metadata file that was not accepted, and why.

type CrashOutcome added in v0.2.0 

type CrashOutcome string

CrashOutcome says what reading one container's crash directory found. 

const (
	// CrashRecordsFound means that the directory was read and held at least
	// one accepted record.
	CrashRecordsFound CrashOutcome = "records"
	// CrashEntriesRejected means that the directory was read, but every
	// metadata file in it was rejected; Omitted says why.
	CrashEntriesRejected CrashOutcome = "rejected"
	// CrashDirectoryEmpty means that the directory was read and held no
	// metadata file. It is not proof that the daemon never crashed.
	CrashDirectoryEmpty CrashOutcome = "empty"
	// CrashDirectoryAbsent means that the container has no crash directory.
	CrashDirectoryAbsent CrashOutcome = "absent"
	// CrashContainerUnavailable means that the container was never created
	// or was removed, with its crash directory.
	CrashContainerUnavailable CrashOutcome = "unavailable"
	// CrashReadFailed means that Docker or the container implementation
	// failed, for example without permission or after the context ended.
	CrashReadFailed CrashOutcome = "failed"
	// CrashArchiveIncomplete means that the archive was malformed, cut off or
	// over a limit; Records holds what was read before.
	CrashArchiveIncomplete CrashOutcome = "incomplete"
)

type CrashRecord added in v0.2.0 

type CrashRecord struct {
	CrashID       string         `json:"crash_id"`
	Path          string         `json:"path"`
	Entity        string         `json:"entity,omitempty"`
	Process       string         `json:"process,omitempty"`
	CephVersion   string         `json:"ceph_version,omitempty"`
	Timestamp     string         `json:"timestamp,omitempty"`
	Done          bool           `json:"done"`
	Posted        bool           `json:"posted,omitempty"`
	Fields        map[string]any `json:"fields"`
	OmittedFields []string       `json:"omitted_fields,omitempty"`
}

CrashRecord is one daemon-written crash metadata file. Fields holds the known Ceph crash fields, redacted; OmittedFields names the others, whose values are not kept. Done reports the done marker with which Ceph completes a crash report. Posted means that the report was in the posted directory, where ceph-crash moves the reports that it sent.

type DaemonAddress added in v0.2.0 

type DaemonAddress struct {
	Name  string
	Addrs []EntityAddress
}

DaemonAddress is the name of one MON or MGR and the public address vector that its cluster map advertises.

type DaemonAddresses added in v0.2.0 

type DaemonAddresses struct {
	MonMapEpoch int
	// Monitors are the monmap members in rank order.
	Monitors    []DaemonAddress
	MgrMapEpoch int
	// ActiveManager is nil when no MGR is active. A MGR that has just become
	// active advertises its addresses only when it is available; until then
	// ManagerAvailable is false and its address vector can be empty. Standby
	// MGRs advertise no address in the MGRMap, so StandbyManagers lists only
	// their names.
	ActiveManager    *DaemonAddress
	ManagerAvailable bool
	StandbyManagers  []string
	OSDMapEpoch      int
	// OSDs are the OSDMap entries in ID order.
	OSDs []OSDAddress
}

DaemonAddresses lists the daemons of a cluster and the addresses that the MON, MGR and OSD maps advertise, with the epoch of each map. The three maps are read one after another, not as one snapshot.

type DiagnosticArtifact 

type DiagnosticArtifact struct {
	Kind        string `json:"kind"`
	Name        string `json:"name"`
	Role        string `json:"role,omitempty"`
	ContainerID string `json:"container_id,omitempty"`
	Data        string `json:"data,omitempty"`
	Truncated   bool   `json:"truncated,omitempty"`
	Error       string `json:"error,omitempty"`
}

DiagnosticArtifact retains one observation, including incomplete output or an individual failure. Data is text; inspect observations are JSON when not truncated. Names and order are stable, but observations are not transactional.

type DiagnosticsConfig 

type DiagnosticsConfig struct {
	Timeout              time.Duration
	OperationTimeout     time.Duration
	MaxOutputBytes       int
	LogTail              int
	Concurrency          int
	MaxContainers        int
	AdditionalContainers []DiagnosticsContainer
	RedactValues         []string
}

DiagnosticsConfig bounds a read-only collection. Zero fields select a one minute total timeout, ten seconds per operation, 64 KiB per artifact, the last 1000 log lines, four workers and at most 128 containers. Additional containers are caller-owned clients or mirror daemons; collection never adopts them. RedactValues removes additional opaque application secrets. Redaction is best effort: arbitrary log contents still require review before sharing a report.

type DiagnosticsContainer 

type DiagnosticsContainer struct {
	Role      string
	Name      string
	Container testcontainers.Container
}

DiagnosticsContainer explicitly adds an external container to a report. Name and Role describe it; Container is only inspected and its logs read.

type DiagnosticsReport 

type DiagnosticsReport struct {
	SchemaVersion int                  `json:"schema_version"`
	StartedAt     time.Time            `json:"started_at"`
	FinishedAt    time.Time            `json:"finished_at"`
	HostNetwork   bool                 `json:"host_network"`
	Closed        bool                 `json:"closed"`
	Complete      bool                 `json:"complete"`
	Artifacts     []DiagnosticArtifact `json:"artifacts"`
}

DiagnosticsReport is an independent, JSON-serializable snapshot. Complete means every requested observation succeeded without byte truncation, not that Ceph is healthy or that replication has completed. Logs cover only LogTail.

type EntityAddress added in v0.2.0 

type EntityAddress struct {
	// Type is the protocol: "v2" for msgr2, "v1" for the legacy protocol, or
	// "any".
	Type string
	// Addr is the IP address and port as Ceph prints it, such as
	// "172.20.0.3:6800" or "[fd00::3]:6800".
	Addr string
	// Nonce tells apart daemon instances that reuse an address. MONs use 0.
	Nonce uint32
}

EntityAddress is one entry of a Ceph address vector, as a cluster map advertises it.

type ErasureCodeConfig 

type ErasureCodeConfig struct {
	K               int
	M               int
	AllowOverwrites bool
}

ErasureCodeConfig selects data (K) and parity (M) shards on separate failure domains using PoolConfig's placement fields. The fixture uses the portable Jerasure reed_sol_van plugin. Replicas must be zero; zero MinSize selects K+1. An explicit MinSize of K permits I/O without any spare shard during a failure. AllowOverwrites enables partial object writes and is required for RBD and CephFS data pools. Their metadata must remain in a replicated pool.

type FullRatioSnapshot 

type FullRatioSnapshot struct {
	FSID   string
	Epoch  uint32
	Ratios FullRatios
}

FullRatioSnapshot is one native OSDMap observation. Epoch is not a generation for these thresholds: unrelated OSDMap changes can advance it too. The values describe map policy, not whether any OSD has reported the corresponding state.

type FullRatios 

type FullRatios struct {
	NearFull, BackfillFull, Full float64
}

FullRatios are global OSDMap utilization thresholds, expressed as fractions of capacity. These are separate from central configuration and pool quotas. Native OSDMap storage uses float32; observations retain its normalized value.

type FullRatiosOverride 

type FullRatiosOverride struct {
	// contains filtered or unexported fields
}

FullRatiosOverride restores all three global OSDMap thresholds as one owned fixture. Copies share lifecycle state. The native CLI changes one threshold per command, so apply and restore are not atomic. A non-nil handle returned with an error remains tracked; restore it or terminate the disposable cluster.

func (*FullRatiosOverride) Restore 

func (change *FullRatiosOverride) Restore(ctx context.Context) error

Restore returns all three thresholds to their captured values. It is idempotent and reconciles uncertain native replies before another mutation. A different current tuple is refused rather than overwriting outside edits. Only the latest owned tuple or the two sides of one uncertain command are accepted; older apply prefixes do not authorize restoration. Unrelated flags, pool policies and central configuration are not changed.

type HealthCheck 

type HealthCheck struct {
	Severity, Summary string
	Count             int64
	Details           []string
	Muted             bool
}

HealthCheck reports the native summary count and detail messages. Count is signed and need not equal the number of details, which Ceph may truncate.

type HealthMute 

type HealthMute struct {
	Code, Summary, ExpiresAt string
	Count                    int64
	Sticky                   bool
}

HealthMute is a native stored mute, which can outlive its current check. ExpiresAt is empty for no TTL, otherwise the original Ceph timestamp: YYYY-MM-DDTHH:MM:SS.ffffff+HHMM (or -HHMM), or seconds.ffffff for an early epoch below 315360000 seconds. Expiry is not compared with the caller's clock; an expired or cleared nonsticky mute can remain visible before the MON tick.

type HealthSnapshot 

type HealthSnapshot struct {
	FSID, Status string
	Checks       map[string]HealthCheck
	Mutes        []HealthMute
}

HealthSnapshot is one MON health observation. Status reflects the worst unmuted check; HEALTH_OK can therefore include muted warnings or errors. Checks retain native, case-sensitive codes, including unknown future codes.

type InconsistentObject 

type InconsistentObject struct {
	Name, Namespace  string
	Errors           []string
	UnionShardErrors []string
	Shards           []InconsistentShard
}

InconsistentObject is one object reported by the last deep scrub of a PG. Errors are object-level native error names and UnionShardErrors combine the per-shard names, such as read_error or data_digest_mismatch.

type InconsistentShard 

type InconsistentShard struct {
	OSD, Shard int
	Primary    bool
	Errors     []string
}

InconsistentShard retains one native shard report. Shard is -1 for replicated pools.

type MGRModuleOverride 

type MGRModuleOverride struct {
	// contains filtered or unexported fields
}

MGRModuleOverride restores one module's original enabled state. Copies share lifecycle state. Restore explicitly when the test condition ends; termination disposes of the entire cluster. A non-nil handle returned with an error must also be restored or the cluster terminated.

func (*MGRModuleOverride) Restore 

func (change *MGRModuleOverride) Restore(ctx context.Context) error

Restore returns only this module to its original membership. It is idempotent and reconciles an uncertain restoration reply before mutating again. It checks dependency metadata before enabling and current filesystem/mirror usage before disabling. No-op leases refuse an outside change. Outside writes equal to the applied or previous boolean state cannot be distinguished and must not race.

type MGRModuleState 

type MGRModuleState struct {
	Name                             string
	Enabled, AlwaysOn, ForceDisabled bool
	Available, CanRun                bool
	ErrorString                      string
}

MGRModuleState combines configured module membership with the active MGR's advertised module metadata. Available means metadata is present, not that commands are ready. CanRun is meaningful only when Available is true. AlwaysOn modules can be force-disabled outside this fixture; Enabled reflects ForceDisabled too. ErrorString is the native dependency/load diagnostic.

type ManagerContainer 

type ManagerContainer struct {
	testcontainers.Container
	DaemonName string
	// contains filtered or unexported fields
}

ManagerContainer is an owned active or standby manager.

type ManagerStatus 

type ManagerStatus struct {
	Available  bool   `json:"available"`
	ActiveName string `json:"active_name"`
	ActiveGID  uint64 `json:"active_gid"`
	Standbys   []struct {
		Name string `json:"name"`
		GID  uint64 `json:"gid"`
	} `json:"standbys"`
}

ManagerStatus reports active ownership and registered standbys.

type MessengerConnectionMode added in v0.2.0 

type MessengerConnectionMode string

MessengerConnectionMode is one Messenger v2 connection mode. 

const (
	// MessengerCRC checks frame integrity with CRC32C and does not encrypt.
	MessengerCRC MessengerConnectionMode = "crc"
	// MessengerSecure encrypts frames with AES-GCM.
	MessengerSecure MessengerConnectionMode = "secure"
)

type MessengerMode 

type MessengerMode int

MessengerMode selects, at bootstrap, the Messenger protocols that every daemon listens on and advertises, and whether the connection modes stay configurable. Both are fixed for the life of the cluster. 

const (
	// MessengerDefault retains Ceph's own Messenger defaults and v1/v2 MON
	// addresses. It does not imply that every connection uses encryption.
	MessengerDefault MessengerMode = iota
	// MessengerV2Secure allows only encrypted Messenger v2 connections, with
	// no CRC-mode negotiation or legacy Messenger v1 listener.
	MessengerV2Secure
	// MessengerV2Only listens on and advertises only Messenger v2, like
	// MessengerV2Secure, but keeps the connection modes configurable: Ceph's
	// defaults, WithMessengerPolicy, TemporaryMessengerPolicy and
	// WithClientMessengerPolicy select them, as with MessengerDefault.
	MessengerV2Only
)

type MessengerPolicy added in v0.2.0 

type MessengerPolicy struct {
	// Monitors sets ms_mon_service_mode for the MONs.
	Monitors []MessengerConnectionMode
	// Managers sets ms_mon_service_mode for the MGRs: Ceph applies the MON
	// option to connections from clients to MGRs.
	Managers []MessengerConnectionMode
	// OSDs sets ms_service_mode for the OSDs.
	OSDs []MessengerConnectionMode
}

MessengerPolicy selects, for each daemon type, the Messenger v2 connection modes that it accepts from clients. The order of a list does not matter: the connection uses the first mode in the client's order of preference that the daemon accepts. A nil field keeps Ceph's default, which accepts both modes. Connections between daemons keep their defaults, so every policy leaves the cluster itself working.

type MessengerPolicyOverride added in v0.2.0 

type MessengerPolicyOverride struct {
	// contains filtered or unexported fields
}

MessengerPolicyOverride holds the configuration overrides of one TemporaryMessengerPolicy call.

func (*MessengerPolicyOverride) Restore added in v0.2.0 

func (override *MessengerPolicyOverride) Restore(ctx context.Context) error

Restore returns every option of the policy to its previous value, in reverse order. It is idempotent and can be retried after a failure.

type MonitorContainer 

type MonitorContainer struct {
	testcontainers.Container
	DaemonName string
}

MonitorContainer is an owned quorum member. Stop/Start preserves its store; RemoveMonitor explicitly removes membership and deletes the container.

type NativeCommand added in v0.2.0 

type NativeCommand struct {
	// Argv is the program and its arguments. It runs without a shell.
	Argv []string
	// Timeout bounds the command's run time, at most
	// MaxNativeCommandTimeout. The caller's context can shorten it.
	Timeout time.Duration
	// MaxStdout and MaxStderr bound each output stream inside the container,
	// each at most MaxNativeCommandOutput bytes.
	MaxStdout, MaxStderr int
	// Label names the command in errors, which never contain its arguments
	// or output. It defaults to "native command".
	Label string
	// Input, when set, is the command's standard input, and MaxInput, at
	// most MaxNativeCommandInput, bounds the bytes written to it in all.
	// Without Input the command's standard input is empty, and MaxInput must
	// be 0. An Input serves one command.
	Input    *NativeInput
	MaxInput int
}

NativeCommand is one native command that RunNativeCommand runs under a supervisor inside a container.

type NativeCommandResult added in v0.2.0 

type NativeCommandResult struct {
	Stdout, Stderr []byte
	// ExitCode is the command's exit code; a nonzero one is not an error.
	ExitCode int
	// TimedOut means that the supervisor killed the command's process group
	// at its deadline; the output is what it wrote until then.
	TimedOut bool
	// StdoutTruncated and StderrTruncated mean that a stream exceeded its
	// limit; the result holds its first bytes.
	StdoutTruncated, StderrTruncated bool
	// InputWritten counts the bytes of Input that the input stream to the
	// container took, and InputDelivered those that the supervisor wrote into
	// the command's standard input; the command may not have read all of
	// them. They differ when the command ended, or closed its standard input,
	// before it took everything. InputEOF means that the command's standard
	// input ended after all InputWritten bytes, because Input was closed.
	InputWritten, InputDelivered int64
	InputEOF                     bool
}

NativeCommandResult is the outcome of a supervised native command. Stdout and Stderr are separate, raw and binary-safe.

func (NativeCommandResult) Complete added in v0.2.0 

func (r NativeCommandResult) Complete() bool

Complete reports whether the command finished within its time and output limits, so that its output is complete.

type NativeCommandUnstoppedError added in v0.2.0 

type NativeCommandUnstoppedError struct {
	// contains filtered or unexported fields
}

NativeCommandUnstoppedError reports a native command that RunNativeCommand could not confirm as stopped after its call ended early: it may still run in its container. Stop retries the confirmation; Terminate also tries it.

func (*NativeCommandUnstoppedError) Error added in v0.2.0 

func (e *NativeCommandUnstoppedError) Error() string

func (*NativeCommandUnstoppedError) Stop added in v0.2.0 

func (e *NativeCommandUnstoppedError) Stop(ctx context.Context) error

Stop stops the command, or confirms that it ended with its container, and returns nil once the command can no longer run: after the container was removed, or once it runs again after a restart. A stopped container cannot confirm anything until then. Each attempt has its own NativeCommandStopTimeout, and waits for a concurrent attempt within ctx. Stop is idempotent and can be retried, also after Terminate.

func (*NativeCommandUnstoppedError) Unwrap added in v0.2.0 

func (e *NativeCommandUnstoppedError) Unwrap() error

type NativeFailure added in v0.2.0 

type NativeFailure struct {
	// Program is the command's program, such as "ceph" or "rbd"; the rest of
	// its arguments is never recorded.
	Program string
	// ExitCode is its exit code.
	ExitCode int
	// Errno is the error code that the tool reported, such as "ENOENT", or
	// "" when it reported none. It is read from the "Error ENOENT:" that
	// starts a line of Ceph and from the last "(2) No such file or directory"
	// of a line of the other tools, never guessed from the exit code.
	Errno string
	// Reason is the tool's own explanation: the lines of its error output
	// without Ceph's log lines, joined with "; ", at most
	// MaxNativeFailureReason bytes, with keys and the secrets that
	// RedactSecrets recognizes replaced. It can name resources of the
	// request, such as a pool.
	Reason string
}

NativeFailure is the status of a native command that ran and exited nonzero. errors.Is matches it with ErrNativeFailure, and with fs.ErrNotExist, fs.ErrPermission or fs.ErrExist when the tool reported ENOENT, EACCES or EPERM, or EEXIST. RunNativeCommand returns a nonzero exit as its result instead; the module's own commands and Ceph return this.

func (*NativeFailure) Error added in v0.2.0 

func (f *NativeFailure) Error() string

func (*NativeFailure) Is added in v0.2.0 

func (f *NativeFailure) Is(target error) bool

Is reports whether target is ErrNativeFailure, or the fs error that matches the reported errno.

type NativeInput added in v0.3.0 

type NativeInput struct {
	// contains filtered or unexported fields
}

NativeInput is the standard input of one NativeCommand while it runs. Write sends data in order, and Close ends the input. The command gets exactly the bytes written, without a shell, an encoding or a terminal in between. Nothing that is written appears in arguments, the environment, files, errors, labels or diagnostics; the command's own output is returned as it wrote it, so a command that must not reveal its input must not echo it.

Inside the container the supervisor reads what arrives at once, up to MaxInput, and passes it on as the command reads, so a command that reads slowly or not at all does not hold up Write. Its methods are safe for concurrent use.

func NewNativeInput added in v0.3.0 

func NewNativeInput() *NativeInput

NewNativeInput returns an open input for one NativeCommand.

func (*NativeInput) Close added in v0.3.0 

func (in *NativeInput) Close() error

Close ends the input: the command gets the end of input after every byte written before. Close is idempotent and never fails; a command that ended before is not affected.

func (*NativeInput) Write added in v0.3.0 

func (in *NativeInput) Write(ctx context.Context, data []byte) error

Write sends data to the command's standard input after the data of earlier writes, and returns once the input stream to the container took all of it. A write before RunNativeCommand starts the command waits for it. ctx bounds that wait; when ctx ends after the stream took the data, Write returns ctx's error, and the command may still get the data, as NativeCommandResult.InputWritten tells. Write copies data before it returns.

A write that would take the input past MaxInput is refused whole with an error that matches ErrInvalidRequest, and the input stays open. Once the input is closed, or the command ended or its input stream failed, Write returns an error that matches ErrNativeInputClosed. An empty write sends nothing.

type NetworkDelay added in v0.2.0 

type NetworkDelay struct {
	// HelperImage is the netem helper image: NetemHelperImage, or another image
	// that meets contract 1 of the netem extension. Required.
	HelperImage string
	// Plane selects the bridge whose endpoint of the target is delayed.
	Plane NetworkPlane
	// Delay is added to every packet that leaves the target through that
	// endpoint: at least 1ms and at most MaxNetworkDelay, in whole
	// microseconds.
	Delay time.Duration
	// Jitter varies each packet's delay uniformly within ±Jitter. It is at
	// most Delay, in whole microseconds.
	Jitter time.Duration
}

NetworkDelay configures TemporaryNetworkDelay.

type NetworkDelayLease added in v0.2.0 

type NetworkDelayLease struct {
	// ContainerID and NetworkName identify the delayed endpoint.
	ContainerID, NetworkName string
	// SandboxID is the Docker network sandbox of the target when the delay
	// was applied, and Namespace the kernel's name of its network namespace.
	SandboxID, Namespace string
	// Interface, InterfaceIndex, MACAddress and Address identify the
	// delayed interface inside that namespace.
	Interface      string
	InterfaceIndex int
	MACAddress     string
	Address        netip.Addr
	// HelperImageID is the image of the helper that applied the delay.
	HelperImageID string
	// Delay and Jitter are the requested values.
	Delay, Jitter time.Duration
	// contains filtered or unexported fields
}

NetworkDelayLease owns one netem delay on a container's network endpoint, applied from a helper container in the target's network namespace that alone has NET_ADMIN. Restore removes the delay and the helper. A non-nil lease returned with an error must be restored: the change may have been applied. Restore is retryable.

func (*NetworkDelayLease) Restore added in v0.2.0 

func (l *NetworkDelayLease) Restore(ctx context.Context) error

Restore removes the delay's qdisc and then the helper. A delay whose interface is gone needs no removal. A root qdisc that someone replaced is refused, not removed, and the lease stays retryable.

func (*NetworkDelayLease) Status added in v0.2.0 

func (l *NetworkDelayLease) Status(ctx context.Context) (NetworkDelayStatus, error)

Status reads the delayed interface's qdiscs through the helper. After the target restarted, stopped, was removed or left the bridge, it reports Gone.

type NetworkDelayStatus added in v0.2.0 

type NetworkDelayStatus struct {
	// Applied means that the delay's own netem qdisc is the root qdisc of the
	// interface it was applied to.
	Applied bool
	// Gone means that the delayed interface no longer exists: the target
	// stopped, restarted, was removed or was disconnected from the network,
	// and the delay ended with it.
	Gone bool
	// Interface is the delayed interface's current name.
	Interface string
	// RootKind and RootHandle describe the interface's current root qdisc.
	RootKind, RootHandle string
	// Delay and Jitter are the kernel's values while Applied.
	Delay, Jitter time.Duration
	// Packets, Bytes and Drops are the netem qdisc's counters while Applied.
	Packets, Bytes, Drops uint64
	// Qdiscs is tc's JSON output for the interface.
	Qdiscs json.RawMessage
}

NetworkDelayStatus is what the kernel reports for a delay's interface, read through the helper. It is native state, not the requested NetworkDelay.

type NetworkInterruption 

type NetworkInterruption struct {
	ContainerID, NetworkName string
	OriginalAddress          netip.Addr
	// contains filtered or unexported fields
}

NetworkInterruption owns one disconnected Docker endpoint. Restore reconnects its original addresses and aliases without restarting the container or Ceph daemon. It is an endpoint-to-network interruption, not a pairwise packet filter. A non-nil handle returned with an error must be restored: transport failure cannot prove that Docker did not disconnect it. Restore is retryable.

func InterruptNetwork 

func InterruptNetwork(ctx context.Context, ctr testcontainers.Container, name string) (*NetworkInterruption, error)

InterruptNetwork disconnects one existing bridge endpoint. The caller owns the container and the returned restoration handle; this function does not stop a daemon, change Ceph policies or remove the network. Fixture-specific helpers also register restoration with their cleanup. Host networking cannot be interrupted through Docker endpoint operations.

func (*NetworkInterruption) Restore 

func (link *NetworkInterruption) Restore(ctx context.Context) error

Restore reattaches the original endpoint, preserving the IPv4 and, on a dual-stack bridge, the IPv6 address advertised by Ceph. Removed containers need no restoration. An externally reattached endpoint with a different address is refused rather than overwritten, and the handle stays retryable.

type NetworkPlane 

type NetworkPlane string

NetworkPlane selects the client/public or OSD replication network. 

const (
	PublicNetworkPlane  NetworkPlane = "public"
	ClusterNetworkPlane NetworkPlane = "cluster"
)

type OSDAddress added in v0.2.0 

type OSDAddress struct {
	ID     int
	Up, In bool
	// Public serves clients; Cluster carries replication and recovery.
	Public, Cluster []EntityAddress
	// HeartbeatFront and HeartbeatBack carry OSD heartbeats on the public and
	// cluster sides.
	HeartbeatFront, HeartbeatBack []EntityAddress
}

OSDAddress is one OSD's entry in the OSDMap. The address vectors are those of the OSD's last boot; a down OSD keeps them.

type OSDConfig 

type OSDConfig struct {
	Host        string
	Rack        string
	Root        string
	DeviceClass string
}

OSDConfig places a container's OSD in the logical CRUSH hierarchy. Root defaults to the cluster's selected default root, Host defaults to osd-ID, and Rack/DeviceClass are optional. Multiple OSDs may share a host or rack for domain-failure tests.

type OSDContainer 

type OSDContainer struct {
	testcontainers.Container
	ID int
	// contains filtered or unexported fields
}

OSDContainer is one storage daemon backed by a sparse BlueStore file. The file is container-local unless WithOSDInMemoryStorage is selected. Stop/Start can be used for failure injection; RemoveOSD drains and purges it.

func (*OSDContainer) Placement 

func (o *OSDContainer) Placement() OSDConfig

Placement returns a copy of this OSD's resolved logical CRUSH location.

type OSDFlagOverride 

type OSDFlagOverride struct {
	// contains filtered or unexported fields
}

OSDFlagOverride restores one global OSDMap flag. Copies share lifecycle state. Restore explicitly when the condition should end; termination disposes of the entire OSDMap. A non-nil handle returned with an error remains tracked and must be restored or the cluster terminated. Unrelated flags are never changed.

func (*OSDFlagOverride) Restore 

func (change *OSDFlagOverride) Restore(ctx context.Context) error

Restore returns only this flag to its original state. It is idempotent and reconciles a lost restoration reply before another native mutation. A no-op lease refuses an outside change rather than overwriting it. Outside writes indistinguishable from applied/prior boolean states cannot be detected.

type OSDObjectStore added in v0.2.0 

type OSDObjectStore string

OSDObjectStore names the object store that OSDs keep their data in. 

const (
	// OSDBlueStore keeps each OSD's data in a sparse BlueStore file. It is the
	// default.
	OSDBlueStore OSDObjectStore = "bluestore"
	// OSDMemStore keeps each OSD's data in the OSD process's memory, with
	// WithOSDBlockSize as its capacity. A clean stop saves the store to the
	// OSD's data directory and Start loads it again; a killed OSD loses it.
	// It is Ceph's test backend and needs no block file.
	OSDMemStore OSDObjectStore = "memstore"
)

type OSDState 

type OSDState struct {
	ID     int
	UUID   string
	Up, In bool
	Weight float64
}

OSDState reports authoritative OSDMap state for an owned OSD. Weight is the native map reweight in [0,1], not the CRUSH capacity weight. Out OSDs retain their containers, identities and data; absence or a replaced UUID is an error.

type ObjectWatcher 

type ObjectWatcher struct {
	Address  string
	ClientID uint64
	Cookie   uint64
}

ObjectWatcher is one client watching a RADOS object. Address is the client's IP:port/nonce, the same form BlocklistEntries reports. A client that stops answering keeps its watch until the OSD's osd_client_watch_timeout expires, 30 seconds by default; until then a notify to the object waits for that client and times out.

type OperationError added in v0.2.0 

type OperationError struct {
	// Kind is the category: ErrInvalidRequest, ErrUnsupported,
	// ErrImageCapability, ErrEnvironmentUnavailable or ErrNativeFailure.
	Kind error
	// Op names the operation, such as "create client", when the error's
	// text does not already start with it.
	Op string
	// Target names the container or cluster that the failure concerns, such
	// as "control", "osd.0", "destination cluster" or "netem helper".
	Target string
	// Version is the Ceph release observed on Target, when it matters.
	Version string
	// Image is the immutable ID of Target's image, when the fixture read it.
	Image string
	// Capability names the missing or unsupported capability, such as
	// "ceph-authtool --key-type" or "python3".
	Capability string
	// Err is the cause.
	Err error
}

OperationError is a classified failure with the details that identify it. Its text never includes command arguments, raw output or credentials.

func (*OperationError) Error added in v0.2.0 

func (e *OperationError) Error() string

func (*OperationError) Is added in v0.2.0 

func (e *OperationError) Is(target error) bool

Is reports whether target is the error's category.

func (*OperationError) Unwrap added in v0.2.0 

func (e *OperationError) Unwrap() error

type Option 

type Option func(*options) error

Option transfers cluster settings outside the monitor's container request.

func WithAddressFamily added in v0.2.0 

func WithAddressFamily(family AddressFamily) Option

WithAddressFamily selects the addresses of the Messenger listeners of every MON, MGR, OSD and service daemon. AddressIPv6 and AddressIPv4MappedIPv6 need bridge mode, and AddressIPv6 a Docker Engine that can allocate IPv6 subnets. With WithSeparateClusterNetwork, AddressIPv6 makes both bridges dual-stack and their IPv6 subnets Ceph's public and cluster networks; AddressIPv4MappedIPv6 cannot be combined with it.

func WithAuth added in v0.2.0 

func WithAuth(config AuthConfig) Option

WithAuth selects the Cephx key type, the MonMap ciphers and the ticket lifetimes of a new cluster. The settings are part of the bootstrap: the key type and ciphers apply from the first MON, and the lifetimes are written to every daemon's and WithClient container's ceph.conf, so TemporaryConfig cannot change them later. An image without key type support, such as Squid or Tentacle before 20.2.4, accepts a configuration in which every cipher is CipherAES, and Run fails before any daemon starts when one is not.

func WithConfigFile 

func WithConfigFile(path string) Option

WithConfigFile merges a native ceph.conf file from the test host into the bootstrap configuration of every daemon and WithClient container, like the WithConfigFile options of testcontainers-go modules. Entries replace the fixture's defaults for the same section and key, and later files override earlier files key by key. Ceph gives local files precedence over the MON configuration database, so TemporaryConfig cannot change a key set here. Keys that the fixture owns or passes on a daemon command line are rejected: identity, addresses, networks, Cephx, Messenger modes, logging, data paths, pool size defaults and BlueStore file layout. Use the dedicated options for those. RGW runs as client.admin, so RGW settings belong in [client] or [client.admin]. Includes, continuations and keys outside a section are rejected.

func WithDefaultCRUSHRoot 

func WithDefaultCRUSHRoot(root string) Option

WithDefaultCRUSHRoot selects placement for pools that Ceph creates itself, including .mgr, and for omitted PoolConfig.CRUSHRoot/OSDConfig.Root fields. The initial layout must contain enough OSDs in this root for the default replica count. Without this option the root is default.

func WithHostAddress 

func WithHostAddress(address string) Option

WithHostAddress selects a local IPv4 address to bind and advertise in host mode, typically the Docker host's address when Docker is remote. The address must be assigned on the Docker daemon host and reachable from the test process; Run verifies both. Requires WithHostNetwork.

func WithHostNetwork 

func WithHostNetwork() Option

WithHostNetwork places every cluster daemon and WithClient container in the Docker daemon's host network. MON and RGW ports are selected automatically. The default advertised address is 127.0.0.1 for clients on that same host. Run fails with ErrHostNetworkUnavailable unless the engine runs Linux containers and the test process reaches the first MON's advertised endpoint.

func WithInitialOSDs 

func WithInitialOSDs(configs ...OSDConfig) Option

WithInitialOSDs selects the count and logical CRUSH locations of initial OSDs. A subsequent WithOSDCount replaces this layout with default OSD locations.

func WithManagerCount 

func WithManagerCount(count int) Option

WithManagerCount starts one active manager and the remaining standbys.

func WithMessengerMode 

func WithMessengerMode(mode MessengerMode) Option

WithMessengerMode selects the Messenger policy at bootstrap. MessengerDefault retains Ceph's image defaults and v1/v2 addresses. MessengerV2Secure requires encrypted v2 connections without CRC-mode or v1 fallback, including later daemons and WithClient containers. MessengerV2Only removes the v1 listeners and addresses in the same way but leaves the connection modes to Ceph's defaults and the Messenger policy options. This option does not enable RGW HTTP TLS.

func WithMessengerPolicy added in v0.2.0 

func WithMessengerPolicy(policy MessengerPolicy) Option

WithMessengerPolicy selects the connection modes that MONs, MGRs and OSDs accept from clients. Run stores the policy in the central configuration right after the first MON starts, before any other daemon, so every daemon starts with it. TemporaryMessengerPolicy changes it on a running cluster. It cannot be combined with MessengerV2Secure, whose local configuration fixes every mode.

func WithMonitorCount 

func WithMonitorCount(count int) Option

WithMonitorCount selects the initial monitor count. Three enables quorum failover tests; even counts are allowed for deliberate quorum-loss scenarios.

func WithNoInitialManagers 

func WithNoInitialManagers() Option

WithNoInitialManagers defers the first MGR until an explicit AddManager. Run waits for MON quorum and requested OSDs up/in, without implying PG clean or MGR module readiness. Initial CephFS/RGW services and an explicit manager count are rejected before allocation; positive-OSD initial pools are allowed. The ordinary default is one MGR and WithManagerCount(0) remains invalid. The control image must still contain the MGR and its required dependencies.

func WithNoInitialOSDs 

func WithNoInitialOSDs() Option

WithNoInitialOSDs starts only the selected MON/MGR topology. It keeps the ordinary prospective pool defaults (size 2, min_size 1) for later AddOSD calls; WithPoolDefaults may override them. Initial pools, CephFS/RGW services and explicit WithOSDCount/WithInitialOSDs options are rejected before allocation. No storage readiness is implied: native internal pools may have pending PGs. WithOSDCount(0) and WithInitialOSDs() keep their existing validation errors.

func WithOSDBlockSize 

func WithOSDBlockSize(size int64) Option

WithOSDBlockSize sets each sparse BlueStore file's logical size in bytes. The minimum is 64 MiB and the default remains 1 GiB. These are disposable files, not real disks. Small functional fixtures can use WithNoInitialOSDs and TemporaryConfig to skip mClock's startup capacity benchmark before AddOSD; the benchmark can exhaust a small file even without client data.

func WithOSDCount 

func WithOSDCount(count int) Option

WithOSDCount sets the initial number of OSD containers (default: 2).

func WithOSDImage 

func WithOSDImage(image string) Option

WithOSDImage selects the image for initial and subsequently added OSDs. If omitted, OSDs use the image passed to Run. Use matching Ceph versions.

func WithOSDInMemoryStorage 

func WithOSDInMemoryStorage(maxBytes int64) Option

WithOSDInMemoryStorage keeps all OSD stores in one Docker-managed tmpfs volume. maxBytes is a positive cluster-wide filesystem allocation ceiling, independent of each sparse file's logical WithOSDBlockSize. It does not reserve RAM or limit daemon RSS; tmpfs may swap, and a full volume fails I/O. An owned helper using the control image preserves data across OSD Stop/Start. Cluster termination discards the volume. Helper loss or Docker host/VM restart can lose data. Requires a Linux Docker engine with local tmpfs volume support, including compatible Docker Desktop engines. No host mount, privileged container or additional image is required. The default remains container-local sparse files. NoInitialOSDs defers allocation until AddOSD.

func WithOSDObjectStore added in v0.2.0 

func WithOSDObjectStore(store OSDObjectStore) Option

WithOSDObjectStore selects the object store of every OSD in the cluster, including OSDs added later. WithOSDBlockSize sets the capacity of a MemStore OSD. MemStore cannot be combined with WithOSDInMemoryStorage, which places BlueStore files in memory, and InjectObjectDataError needs BlueStore.

func WithPoolDefaults 

func WithPoolDefaults(replicas, minSize int) Option

WithPoolDefaults selects size/min_size for pools created by Ceph or callers. Without this option the fast fixture uses size=min(2, OSD count), min_size=1. CreatePool can override these defaults for each explicitly configured pool.

func WithPools 

func WithPools(configs ...PoolConfig) Option

WithPools creates the selected pools after initial OSD startup. Repeated options replace the list. Configs and nested EC settings are copied; cluster defaults are resolved after all Run options have been applied.

func WithSeparateClusterNetwork 

func WithSeparateClusterNetwork() Option

WithSeparateClusterNetwork creates a second owned bridge for OSD replication, recovery and heartbeat traffic. MON/MGR/MDS/RGW and WithClient use only the public bridge. Docker selects both subnets; with WithAddressFamily(AddressIPv6) both bridges are dual-stack and Ceph uses their IPv6 subnets. This option requires bridge mode.

func WithStartupTimeout 

func WithStartupTimeout(timeout time.Duration) Option

WithStartupTimeout bounds bootstrap, readiness and topology operations. Each operation is also bounded by the caller's context. Default: 3 minutes.

func (Option) Customize 

func (Option) Customize(*testcontainers.GenericContainerRequest) error

Customize implements testcontainers.ContainerCustomizer.

type PGState 

type PGState struct {
	PGID, State                                 string
	Up, Acting                                  []int
	UpPrimary, ActingPrimary                    int
	ReportedEpoch, MappingEpoch, LastEpochClean uint32
	ReportedSequence                            uint64
	StatsInvalid                                bool
	Stats                                       PGStats
}

PGState retains native reported placement and statistics. Up and Acting preserve order, empty vectors and repeated missing EC shard slots (CRUSH_ITEM_NONE, 2147483647). Primaries are explicit reported values, not inferred from the vectors; -1 means no primary. Epochs can be zero or older than the OSDMap brackets, and State retains unknown or future native states. A reported OSD may have been removed or be outside fixture ownership.

type PGStats 

type PGStats struct {
	Objects, Bytes, ObjectCopies                      int64
	ObjectsDegraded, ObjectsMisplaced, ObjectsUnfound int64
}

PGStats contains signed native PG counters. Bytes counts logical payload, not physical disk allocation; ObjectCopies is separate from logical Objects. Reports arrive asynchronously, including when StatsInvalid is false, so these counters are not an I/O visibility or redistribution barrier.

type Pool 

type Pool struct {
	PoolConfig
	CRUSHRule          string
	ErasureCodeProfile string
}

Pool identifies the pool, CRUSH rule and optional EC profile created for a fixture. PoolConfig contains the resolved defaults. Cluster termination disposes of its data together with the rest of the cluster.

type PoolConfig 

type PoolConfig struct {
	Name        string
	PGNum       int
	Replicas    int
	MinSize     int
	Application string
	// FailureDomain selects distinct osd, host or rack buckets. Empty uses osd.
	FailureDomain string
	// CRUSHRoot selects the hierarchy root; empty uses the cluster's default
	// root, which is default unless WithDefaultCRUSHRoot selects another.
	// DeviceClass optionally limits placement to OSDs explicitly configured
	// with that class.
	CRUSHRoot   string
	DeviceClass string
	ErasureCode *ErasureCodeConfig
	// NoHashPSPool clears the pool's hashpspool flag. Ceph then computes each
	// PG's placement seed with the legacy additive formula, the PG number plus
	// the pool ID, instead of hashing the PG number with the pool ID, as for
	// pools that predate the flag. It is set right after the pool is created,
	// before any data is written.
	NoHashPSPool bool
}

PoolConfig describes a pool in the disposable, file-backed cluster. Zero PGNum selects 8 PGs with autoscaling disabled. A nil ErasureCode creates a replicated pool, with the cluster's replica/min-size defaults when omitted (normally 2/1, or 1/1 for one initial OSD). Application can be rados, rbd, cephfs, rgw or a custom name; empty leaves it unassigned.

type PoolPGSnapshot 

type PoolPGSnapshot struct {
	FSID                                string
	PoolBefore, PoolAfter               PoolState
	OSDMapEpochBefore, OSDMapEpochAfter uint32
	PGReady                             bool
	PGs                                 []PGState
}

PoolPGSnapshot contains one pool's reported PGs, bracketed by two native OSDMap observations. PoolBefore and PoolAfter reuse PoolState policy fields; their names and IDs agree, but policies and OSDMap epochs may differ. These reads are not an atomic placement snapshot. PGReady is the native MGR flag: false can accompany reports, and true can still accompany unknown PGs.

type PoolPlacement 

type PoolPlacement struct {
	FailureDomain string
	CRUSHRoot     string
	DeviceClass   string
}

PoolPlacement selects the CRUSH hierarchy of a replicated pool after creation. Its fields have PoolConfig's meaning and defaults: empty FailureDomain uses osd, empty CRUSHRoot uses default and empty DeviceClass leaves class unset.

type PoolQuota 

type PoolQuota struct {
	MaxBytes   uint64
	MaxObjects uint64
}

PoolQuota limits logical bytes and object count. Zero removes that limit. Enforcement depends on asynchronously reported PG statistics, so the quota is not a strict per-write accounting boundary.

type PoolState 

type PoolState struct {
	ID                   int64
	Name                 string
	Type                 string
	Size, MinSize, PGNum int
	PGNumTarget          int
	PGNumPending         int
	PGPlacementNum       int
	PGPlacementNumTarget int
	CRUSHRule            int
	AutoscaleMode        string
	ErasureCodeProfile   string
	Flags                string
	Quota                PoolQuota
}

PoolState is an authoritative snapshot of a native pool, including pools created by services or through Ceph. Size is replica count or EC shard count. It does not change when a policy is subsequently updated. PGNum is the current count; the target and pending fields show a split or merge still in progress and are zero when the native map omits them.

type PoolUsageSnapshot 

type PoolUsageSnapshot struct {
	FSID, Name         string
	ID                 int64
	StoredBytes        uint64
	StoredDataBytes    uint64
	StoredOMAPBytes    uint64
	AllocatedBytes     uint64
	AllocatedDataBytes uint64
	AllocatedOMAPBytes uint64
	Objects            uint64
	MaxAvailableBytes  uint64
	UsedRatio          float64
}

PoolUsageSnapshot is one monitor-reported pool usage observation. StoredBytes estimates logical user data (DATA + OMAP); AllocatedBytes includes replication or EC and allocation overhead, but not the BlueStore database. All sizes are bytes, Objects is a logical count, and UsedRatio is a fraction in [0, 1]. MaxAvailableBytes estimates writable logical capacity. PG reports arrive asynchronously: this is neither a read-after-write barrier nor a quota check.

type QuorumStatus 

type QuorumStatus struct {
	QuorumNames []string `json:"quorum_names"`
	MonMap      struct {
		FSID string `json:"fsid"`
		Mons []struct {
			Name        string `json:"name"`
			PublicAddrs struct {
				Addrvec []struct {
					Type string `json:"type"`
					Addr string `json:"addr"`
				} `json:"addrvec"`
			} `json:"public_addrs"`
		} `json:"mons"`
	} `json:"monmap"`
}

QuorumStatus contains the actual quorum and advertised monitor addresses.

type Status 

type Status struct {
	FSID   string `json:"fsid"`
	Health struct {
		Status string `json:"status"`
	} `json:"health"`
	MgrMap struct {
		Available bool `json:"available"`
	} `json:"mgrmap"`
	OSDMap struct {
		NumOSDs   int `json:"num_osds"`
		NumUpOSDs int `json:"num_up_osds"`
		NumInOSDs int `json:"num_in_osds"`
	} `json:"osdmap"`
	PGMap struct {
		NumPGs     int `json:"num_pgs"`
		PGsByState []struct {
			StateName string `json:"state_name"`
			Count     int    `json:"count"`
		} `json:"pgs_by_state"`
	} `json:"pgmap"`
}

Status contains the subset of ceph status JSON used for readiness. Use Ceph for other commands or the full JSON response.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.