TL;DR Kafka Metrics | Kafka Summit London
0 likes404 views
The document discusses key metrics for monitoring Kafka clusters, including broker, controller, and client metrics essential for performance assessment. It highlights specific metrics such as under-replicated partitions, request rates, and network performance indicators to ensure efficient cluster operation. Additionally, it mentions external monitoring tools and references for further expertise in Kafka metrics and monitoring.
![18
connection-count
kafka.[producer|consumer]:type=[producer|consumer]-metrics,client-id=([-.w]+)
Client metrics
Alert metrics](https://image.slidesharecdn.com/lt07-20240319-ksl24-redhat-selengegantigmaa-240402161057-af7a7826/85/TL-DR-Kafka-Metrics-Kafka-Summit-London-18-320.jpg)
![19
incoming|outgoing-byte-rate
kafka.[producer|consumer]:type=[producer|consumer]-metrics,client-id=([-.w]+)
Client metrics
Alert metrics](https://image.slidesharecdn.com/lt07-20240319-ksl24-redhat-selengegantigmaa-240402161057-af7a7826/85/TL-DR-Kafka-Metrics-Kafka-Summit-London-19-320.jpg)
TL;DR Kafka Metrics | Kafka Summit London
- 1. TL;DR Kafka Metrics Gantigmaa Selenge
- 2. 2 TL;DR Kafka Metrics Gantigmaa Selenge
- 3. Kafka Cluster 3 Controller Broke r Client application Consumer Producer Kafka cluster Overview
- 4. Broke r 4 Broker metrics Kafka Cluster Client application Consumer Producer Controller
- 7. 7 BytesInPerSec | BytesOutPerSec kafka.server:type=BrokerTopicMetrics,name={BytesInPerSec|BytesOutPerSec} Cluster Performance Metrics to monitor ReplicationBytesInPerSec | ReplicationBytesOutPerSec kafka.server:type=BrokerTopicMetrics,name={ReplicationBytesInPerSec|ReplicationBytesOutPerSec}
- 9. 9 Unbalanced cluster Metrics to monitor PartitionCount | LeaderCount kafka.server:type=ReplicaManager,name=PartitionCount|LeaderCount
- 12. 12 RequestQueueTimeMs | RequestQueueSize kafka.network:type=RequestChannel,name=RequestQueueTimeMs|RequestQueueSize Slow network Metrics to monitor
- 17. Kafka Cluster 17 Controller Broke r Client application Consumer Producer Client Metrics External monitoring for clusters
- 20. Client application Kafka Cluster Producer JVM JVM 20 Broker Controller Monitoring tools How does it all fit together? Prometheus jmx_exporter server jmx_exporter server Alert Manager PagerDuty Grafana Consumer
- 22. 22 Tuesday 2:00 PM - 2:45 PM Breakout Room 4 Getting the Balance Right with Kafka Connect Kate Stanley 5:30 PM - 6:15 PM Breakout Room 7 Safeguarding Your Kafka Data with Encryption-at-rest Tom Bentley Red Hat Sessions Wednesday 2:00 PM - 2:45 PM Breakout Room 3 Meet the Apache Kafka Committers Tom Bentley 2:15 PM - 3:00 PM Meetup Hub Running Kafka on Kube the Native Way with Operators (plus Kafka Connect book signing) Kate Stanley