1

I have 4 partitions and 4 consumers(A,B,C,D for example). How to configure which consumer will read from which partition using consumer groups. I am using Kafka with Spring boot.

Sammy
  • 336
  • 1
  • 12
souvikc
  • 991
  • 1
  • 10
  • 25
  • please provide complete information. Whether consumer are in same consumer group? Have you read about kafka assigner classes? – Sammy Feb 04 '19 at 11:59
  • Consumers are divided into 2 groups each having one consumer. – souvikc Feb 04 '19 at 13:22
  • So you want to ensure that the four consumers each get a different partition? Or you want each specific consumer to get a specific pre-determined partition? – Robin Moffatt Feb 04 '19 at 13:46
  • Just see if below thread can help you. https://stackoverflow.com/questions/46492707/consumer-how-to-specify-partition-to-read-kafka – whysoseriousson Feb 04 '19 at 14:12

1 Answers1

1

By default, kafka will automatically assign the partitions; if you have 4 consumers in the same group, they will eventually get one partition each. There are properties to configure kafka so it won't immediately do the allocation while you bring up your consumers.

You can also assign the partitions yourself.

Using

public ContainerProperties(TopicPartitionInitialOffset... topicPartitions)

if you are building the container yourself, or

@KafkaListener(id = "baz", topicPartitions = @TopicPartition(topic = "${topic}",
            partitions = "${partition}"))

if you are using @KafkaListener.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179