2

How to find the current controller ID, preferably using command-line, on a Kafka cluster which is using Kraft.

Kafka Version: 3.3

Dojo
  • 5,374
  • 4
  • 49
  • 79

2 Answers2

3

Probably you mean Active Controller ID.

Kafka 3.3 comes with the kafka-metadata-quorum tool.

> bin/kafka-metadata-quorum.sh --bootstrap-server  broker_host:port describe --status
ClusterId:              fMCL8kv1SWm87L_Md-I2hg
LeaderId:               3002
...

Docs: https://kafka.apache.org/documentation/#kraft_metadata_tool

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Cervisiam
  • 31
  • 3
  • I tried about four thrid-party Kafka UIs and they all are reporting a different Active Controller ID on every refresh - even one second apart. Whereas the above command is reporting the same Leader ID consistently, which is what I would expect. But why would the UIs report it differently? Is there a way I can check when the leader/active controller was elected? – Dojo Nov 01 '22 at 04:26
1

When using KRaft, a cluster no longer has a single controller. Instead, nodes in the cluster that are running with the "controller" role, all take part in the controller metadata quorum.

The reason that tools are reporting seemingly random IDs is because of an intentional choice to return a random controller participant ID in the existing metadata APIs. This helps in distributing load equally on the nodes participating in the quorum.

Underneath, the participants of the metadata quorum are maintaining a special topic that is replicated with the Raft consensus algorithm. This topic has a leader and you can get the leader ID of that topic. But it is important to note that this is not equal to the controller on a ZK-backed cluster, that role no longer exists when running with KRaft, and as mentioned is now instead a role shared by many nodes.

You should be able to fetch the current leader ID of a cluster by requesting metadata for the __cluster_metadata topic, or as suggested in another answer by using the kafka-metadata-quorum script.

antonagestam
  • 4,532
  • 3
  • 32
  • 44
  • Actually, controller_id is not even necessarily a quorum participant, but any broker in the cluster I think. I'll update this answer once I find the time. – antonagestam Dec 24 '22 at 13:21