1

So I have a consumer group running and listening to messages from a topic. How do I add more instances of consumers to the consumer group? If this is how i listen to the messages

let consumer = new kafka.ConsumerGroup(options, topicname);

consumer.on("message", function(message) {
    //process message here
  });

I tried creating consumers and including the same groupname in the options,

new kafka.Consumer(
      client,
      [
        { topic: topicName}
      ],
      {
        groupId: groupId
      }
      );

but i couldn't see that being reflected in the consumer group when i run the command

bin/kafka-consumer-groups.sh --describe --group groupname --bootstrap-server localhost:9092

I see the same consumer being assigned different partitions of the topic.

How do I add more consumers to this existing consumer group that was created?

Mister Spurious
  • 271
  • 2
  • 11

1 Answers1

0

In a consumer group, we can not have two consumers assigned to the same partition. We can have one consumer consuming from multiple partitions but vice versa is not true. If you have consumers more than the number of partitions then, in that case, the rest of the consumers will sit idle and will not perform any task.

If the partiton count is more than the number of consumer and when you add more consumers in the same consumer group, zookeeper automatically balances the partitions assigned to the consumers and it will ensure that no two consumer have the same partition else all the message will be processed twice.

Ravi
  • 73
  • 5