2

Say I have a topic T1 with 3 partitions i.e. P1,P2 and P3. Where p1 is leader and rest are followers.

Now there are 2 producers want to push to same topic T1. I believe P1 will be leader for both of them ? Also single offset will be maintained for both of them or offset is maintainer per partition per producer ?

Now I have single consumer which is polling from T1. Will it get messages from both producers by default or it has to explicitly mention producer name if it wants message from specfic producer ?

user3198603
  • 5,528
  • 13
  • 65
  • 125

2 Answers2

4

Leader is not dependent on the producers or consumers, so p1 will be always returned as a leader. Offsets are not important for producers, they are defined per consumer group. Offset determines, which messages were read and committed by a consumer group.

Consumer will always read all the messages, it does not matter, which producer published them.

Katya Gorshkova
  • 1,483
  • 9
  • 16
4

You're maybe mixing up replicas and partitions. When you say you have a topic with 3 partitions, it means your records will be dispatched amongs them according to the record key ( or dispatcher algo) . There is no ' leader partition' . However you have a leader broker that handle a partition. In your case you will have 3 leaders, each of them managing one of your 3 partitions.

An interstingng post here, regarding Kafka partitions:

Understanding Kafka Topics and Partitions

Yannick

Yannick
  • 1,240
  • 2
  • 13
  • 25