1

I have 3 nodes where Kafka is installed. All these 3 nodes have their own zookeeper instances. Are 3 zookeeper instances required or is 1 zookeeper instance suffice? Should we have multiple zookeeper instances for fault tolerance & in such a scenario would one of the instances act as primary and would others be replica?

Punter Vicky
  • 15,954
  • 56
  • 188
  • 315

1 Answers1

3

I'm not sure what you mean by "All these 3 nodes have their own zookeeper instances" Basically you should have a single cluster of one, three or five Zookeeper instances and all Kafka brokers should use the same cluster. You don't need more than one Zookeeper instance but I'd highly recommend to use three or five instances because of availability. We use three instances of Zookeeper to run our Kafka cluster.

Lukáš Havrlant
  • 4,134
  • 2
  • 13
  • 18
  • Sorry , we have one cluster with 3 zookeeper instances. Would one of these instances act as primary and are other failover instances? – Punter Vicky Feb 22 '16 at 19:45
  • Yes, Zookeepers elects a leader and the other two instances acts as followers (=replicas). If the leader fails the new leader is elected and the rest of the world should reconnect to the new leader. – Lukáš Havrlant Feb 22 '16 at 19:51
  • Thanks , is the leader election specific to zookeeper? I thought it was for electing a leader for each kafka partition , which I came across that term recently. – Punter Vicky Feb 22 '16 at 19:55
  • Kafka uses Zookeeper to elect leaders for partitions but Zookeeper itself elects a leader between ZK nodes. You can find out which node is leader or follower by using telnet: `telnet zookeeper1 2181` and then write `stat` and submit. You should see `Mode: follower` or `Mode: leader`. (`zookeeper1` is host name of your ZK node, use whatever domain/IP you're using) – Lukáš Havrlant Feb 22 '16 at 20:07
  • Thanks again , Lukas. One last question - I saw that Kafka has in-sync replicas and one of them can be elected a leader. Does this hold good for Zookeeper as well? – Punter Vicky Feb 22 '16 at 20:27
  • The term In-Sync Replica is pure Kafka term, but some similar synchronizing and replicating logic is also in Zookeeper. Zookeeper is basically a distributed synchronized log. But I don't remember details, sorry :). This looks like an answer you are looking for: http://stackoverflow.com/questions/3662995/explaining-apache-zookeeper – Lukáš Havrlant Feb 22 '16 at 21:30
  • Thanks a lot , Lukas – Punter Vicky Feb 22 '16 at 21:56