0

I have create some KafkaConsumer, next call subscribe on it (using Pattern), and next poll. I noticed that if my consumer running, and later I create some new topic (matches to pattern) this consument will not consume this data! If I restart my app then consumer get data from newly created topics. It is ok ? How can I solve it ?

user1511848
  • 35
  • 7
  • Yes. thanks a lot. After reading that topic i have two doubts: I dont understand why someone suggest seting seekToBeginning and I think about costs of decreasing this value - I set it to 1s. – user1511848 Mar 03 '21 at 07:53
  • Sorry, I did not understand the comment you are reffering to and to be honest also not your comment :) If you have a new question it might be more efficient to start a new question on Stackoverflow instead of dicussing a comment from another answer in this question. – Michael Heil Mar 03 '21 at 08:05
  • Or even better: Ask your question directly under the comment you are unclear. I know you do not have enough reputation to comment, but you will get there soon! – Michael Heil Mar 03 '21 at 08:06
  • Ok, no matter about that. As I understand it is important to change matadata.max.age.ms and set auto.offset.reset to earliest. – user1511848 Mar 03 '21 at 08:17

1 Answers1

1

The KafkaConsumer will do a refresh of meta data of its subscriptions based on the KafkaConsumer configuration metadata.max.age.ms which defaults to 5 minutes.

You could reduce this configuration to have your consumer consuming also newly created topics matching your pattern.

Michael Heil
  • 16,250
  • 3
  • 42
  • 77