Monitor Kafka with Prometheus + Grafana

Monitoring Kafka

Yolean Kafka is one of the simplest Kubernetes container-based Kafka solution in the market to get started with.

Yolean Kafka JMX Metrics Prometheus Exporter

This following adds a sidecar to the broker pods that exports selected JMX metrics over HTTP in a format that Prometheus understands. To add a container to an existing pod we must use the patch command.

Clone repo:

git clone https://github.com/Yolean/kubernetes-kafka.git kafka

Apply patch:

kubectl --namespace kafka patch statefulset kafka -p 
"$(cat kafka/prometheus/50-kafka-jmx-exporter-patch.yml)"

Prometheus Metrics Exporter sidecar

kubectl --namespace kafka apply -k kafka/consumers-prometheus/

Grafana Dashboard

You can import our suggested Grafana dashboards and modify them as you wish:

https://grafana.com/dashboards/10083 (Kafka Minion Dashboard ID 10083)

https://grafana.com/dashboards/10466 (Kafka Minion OPS Dashbaord ID 10466)

For more information visit: