0

Very recently we started to get MessageSizeTooLargeException on the metadata, so we enabled offsets.topic.compression.codec=1, to enable gzip compression, but the overall bytes in rate/messages in rate to the broker hasnt changed. Am i missing something? Is there some other property which needs to be changed?

How does this codec work?

Do we need to add some property on consumers and producers as well? I have just enabled this on the broker.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
neoeahit
  • 1,689
  • 2
  • 21
  • 35

1 Answers1

0

offsets.topic.compression.codec is only for the internal offset topic (namely __consumer_offsets). It is no use for those user-level topics.

See Kafka: Sending a 15MB message on how to avoid MessageTooLargeException/RecordTooLargeException.

Community
  • 1
  • 1
amethystic
  • 6,821
  • 23
  • 25