0

Recently, I started to study Kafka and have been thinking how to adopt it into my service. Some of my messages should be processed in strict order, so I chose to use a key for partitioning on producer. However, even though we just need one partition right now, we might increase the number of partitions in the near future. So, in Kafka, if I increase the number of partitions in a topic then will consumers get messages in order?

Thanks in advance.

Woonggeun Jang
  • 227
  • 1
  • 2
  • 10
  • 1
    [This](https://stackoverflow.com/questions/61526124/how-to-add-partition-to-kafka-topic-and-keep-same-key-message-in-same-partition/61530604#61530604) might be helpful. – Michael Heil Apr 07 '21 at 08:23

1 Answers1

1

If you increase partitions, there's no guarantee that future, equal keys will land in their prior partition, so you'll experience a temporary period, based on topic retention, where you'll have keys spanning more than one partition (by default)

One workaround is to ensure you've consumed all messages, stop all clients interacting with the topic, then empty the topic and increase the count

Or you can start with an increased count to begin with and continue having all equal keys distributed over multiple partitions

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245