Documentation
¶
Overview ¶
Package kafka provides a durable Kafka mutation publisher and consumer.
Index ¶
- func EnsureTopics(ctx context.Context, opts TopicOptions) error
- func ReadDeadLetters(ctx context.Context, selection DeadLetterRange) ([]*kgo.Record, error)
- func ValidateDeadLetterReplay(records []*kgo.Record, store string) ([]queue.Mutation, error)
- type DeadLetterRange
- type Handler
- type Publisher
- type PublisherOptions
- type TopicManager
- type TopicOptions
- type Worker
- type WorkerOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EnsureTopics ¶
func EnsureTopics(ctx context.Context, opts TopicOptions) error
func ReadDeadLetters ¶
ReadDeadLetters uses explicit positions and never joins or advances a group. Missing/expired offsets fail instead of silently inspecting a different range.
Types ¶
type DeadLetterRange ¶
type Publisher ¶
type Publisher struct {
// contains filtered or unexported fields
}
func NewPublisher ¶
func NewPublisher(opts PublisherOptions) (*Publisher, error)
func (*Publisher) Publish ¶
func (p *Publisher) Publish(ctx context.Context, req queue.PublishRequest) (queue.PublishResponse, error)
type PublisherOptions ¶
type PublisherOptions struct {
Store string
Topics *TopicManager
Brokers []string
Topic string
ClientOptions []kgo.Opt
Metrics *sinkmetrics.Metrics
MaxRecordBytes int
MaxBufferedBytes int
}
type TopicManager ¶
type TopicManager struct {
// contains filtered or unexported fields
}
TopicManager gates one store until its durable topic policy has been applied. Each store reconciles independently so an unavailable cluster does not prevent other stores from starting. The closed channel also publishes readiness safely.
func NewTopicManager ¶
func NewTopicManager(options TopicOptions) *TopicManager
func (*TopicManager) Run ¶
func (m *TopicManager) Run(ctx context.Context)
type TopicOptions ¶
type Worker ¶
type Worker struct {
// contains filtered or unexported fields
}
func NewWorker ¶
func NewWorker(opts WorkerOptions) (*Worker, error)
type WorkerOptions ¶
type WorkerOptions struct {
Admission *backpressure.Controller
Memory *capacity.Guard
Topics *TopicManager
Brokers []string
Store string
Topic string
GroupID string
DeadLetterTopic string
Handler Handler
MaxPollRecords int
MaxRetryAttempts int
RetryBackoff time.Duration
MaxRetryBackoff time.Duration
ClientOptions []kgo.Opt
Metrics *sinkmetrics.Metrics
ProcessingTimeout time.Duration
MaxRecordBytes int
ShutdownTimeout time.Duration
}
Click to show internal directories.
Click to hide internal directories.