2

Are there any best practices when selecting value of auto.commit.interval.ms?

I read here that:

In general, it is not recommended to keep this interval too small because it vastly increases the read/write rates in zookeeper and zookeeper gets slowed down because it's strongly consistent across its quorum.

What is too small? Is this still an issue with kafka >= 0.9.0 version?

Community
  • 1
  • 1
Erik Pettersson
  • 399
  • 1
  • 6
  • 14

1 Answers1

5

The question is not what is small, but rather what can you live with? If you can live with re-processing several minutes of messages in case of a crash of your consumer, you could set the interval to a few minutes. Because this is what it is all about: messages will be processed after a (re)start from the last committed offset.

Harald
  • 4,575
  • 5
  • 33
  • 72