0

I am using a 3 broker kafka. If I stop the third node: no problem, I can consume LOGS from node 1 or 2. If I stop the second node : no problem, I can consume LOGS from node 1 or 3. If I stop the first node : I am not able to consume anything.

When I restart kafka-runner service . Logs are here and was wrote properly.

my topic configuration is: Topic: log_topic TopicId: xxx PartitionCount: 32 ReplicationFactor: 3 Configs: segment.bytes=1073741824,retention.ms=7200000 Topic: log_topic Partition: 0 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 1 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 2 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 3 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 4 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 5 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 6 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 7 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 8 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 9 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 10 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic: log_topic Partition: 11 Leader: 3 Replicas: 3,1,2 Isr: 2,3,1 Topic: log_topic Partition: 12 Leader: 3 Replicas: 1,3,2 Isr: 2,3,1 Topic: log_topic Partition: 13 Leader: 2 Replicas: 2,1,3 Isr: 2,3,1 Topic: log_topic Partition: 14 Leader: 3 Replicas: 3,2,1 Isr: 2,3,1 Topic: log_topic Partition: 15 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 16 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic: log_topic Partition: 17 Leader: 3 Replicas: 3,1,2 Isr: 2,3,1 Topic: log_topic Partition: 18 Leader: 3 Replicas: 1,3,2 Isr: 2,3,1 Topic: log_topic Partition: 19 Leader: 2 Replicas: 2,1,3 Isr: 2,3,1 Topic: log_topic Partition: 20 Leader: 3 Replicas: 3,2,1 Isr: 2,3,1 Topic: log_topic Partition: 21 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 22 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic: log_topic Partition: 23 Leader: 3 Replicas: 3,1,2 Isr: 2,3,1 Topic: log_topic Partition: 24 Leader: 3 Replicas: 1,3,2 Isr: 2,3,1 Topic: log_topic Partition: 25 Leader: 2 Replicas: 2,1,3 Isr: 2,3,1 Topic: log_topic Partition: 26 Leader: 3 Replicas: 3,2,1 Isr: 2,3,1 Topic: log_topic Partition: 27 Leader: 2 Replicas: 1,2,3 Isr: 2,3,1 Topic: log_topic Partition: 28 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic: log_topic Partition: 29 Leader: 3 Replicas: 3,1,2 Isr: 2,3,1 Topic: log_topic Partition: 30 Leader: 3 Replicas: 1,3,2 Isr: 2,3,1 Topic: log_topic Partition: 31 Leader: 2 Replicas: 2,1,3 Isr: 2,3,1

When I stop leader node. leader changes properly.

Any Idea?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
TRoupie
  • 3
  • 1

2 Answers2

0

Make sure your __consumer_offsets topic has three replicas and is not only hosted on broker 1. By default it only has 1 replica, as established in server.properties

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
0

This seems like a configuration issue with the __consumer_offsets topic.

_consumer_offsets is used to store information about committed offsets for each topic:partition per group of consumers (groupID)

Whenever a consumer in a group reads data from kafka, it periodically stores the information on the point till the data is read in a special topic called __consumer_offsets. This is done so that the consumer can resume from its last position in case of failure.

Issue

The replication factor for __consumer_offsets topic would have been set to 1 and all the partitions for the topic would also be present in broker 1.

You can describe the __consumer_offsets topic to verify above. If this is the case, then the easiest way to resolve the issue would be to delete the __consumer_offsets topic and recreate it with the desired replication factor.

If deletion is not an option, then following the instructions in the thread below

How to change the number of replicas of a Kafka topic?

Rahul Vaid
  • 98
  • 5
  • Fixed: __consumer_offsets was setup to one and was on first node. I will check who to define this in server properties. I changed attribute with execute command. Thanks a lot – TRoupie Feb 08 '22 at 13:49