Cant send large kafka message

Question

I'm attempting to send a 10770128 byte message over Kafka. I am receiving the following error:

org.apache.kafka.common.errors.RecordTooLargeException: The message is 10770128 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.

I've gone through and added the following settings to server.properties, producer.properties, and consumer.properties.

replica.fetch.max.bytes =12000000
message.max.bytes=12000000
max.message.bytes=12000000
max.request.size=12000000
max.message.max.bytes=12000000
max.partition.fetch.bytes=12000000

this is a three node kafka cluster. Each node has the above settings in each file. Any help is appreciated. Thank you :)

Edit: I have already attempted the configurations seen Here, yet the problem persists

I assume you restarted the cluster after editing the property files? — OneCricketeer, Mar 22 '19 at 22:39
@cricket_007 Yes, I restarted all the broker nodes. I did not restart the zookeeper nodes. I also have configured all the settings in the possible duplicate that you mentioned. — Austin Brinegar, Mar 25 '19 at 16:06
Don't need to edit Zookeeper. I assume you are using a regular Java client, no Spring Boot, Spark, etc? — OneCricketeer, Mar 25 '19 at 19:14
The producer is a spring boot app, however there are no producer settings anywhere on the project. — Austin Brinegar, Mar 25 '19 at 20:27
But you mentioned you have a `producer.properties`? In any case, you will have to change the producer settings within that project — OneCricketeer, Mar 25 '19 at 21:13
the producer.properties exists in the kafka configs directory. I changed the settings in there for posterity. — Austin Brinegar, Mar 25 '19 at 21:49
Sure, but that is meant for `kafka-console-producer`, not any application that happens to run on the same machine — OneCricketeer, Mar 25 '19 at 22:08

Cant send large kafka message

0 Answers0