3

I have Kafka instances running in 2 different VMs. I'm able to send messages to Kafka running in vm-1 using spring kafka-template, but while sending a message to kafka running in vm-2, I'm getting the exception below:

2018-04-19 15:12:57 [kafka-producer-network-thread | producer-1] ERROR o.s.k.s.LoggingProducerListener - Exception thrown when sending a message with key='xxxx' and payload='{79, 98, 106, 1, 4, 22, 97, 118, 114, 111, 46, 115, 99, 104, 101, 109, 97, -28, 51, 123, 34, 116, 12...' to topic xxxxxx-v1: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for xxxxxx-v1-3: 60043 ms has passed since batch creation plus linger time

Producer configuration 

config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, x.x.x.x:port);
config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class);
config.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, 60000);
config.put(ProducerConfig.BATCH_SIZE_CONFIG, 1048576);
config.put(ProducerConfig.LINGER_MS_CONFIG, 100);
config.put(ProducerConfig.MAX_REQUEST_SIZE_CONFIG, 10485760);
config.put(ProducerConfig.ACKS_CONFIG, "all");

I can telnet to both VMs and also ping them.

Mickael Maison
  • 25,067
  • 7
  • 71
  • 68
Ajey kumar HB
  • 31
  • 1
  • 5
  • Possible duplicate of https://stackoverflow.com/q/46750420 – hongsy Jan 04 '21 at 13:27
  • Does this answer your question? [Kafka Producer error Expiring 10 record(s) for TOPIC:XXXXXX: 6686 ms has passed since batch creation plus linger time](https://stackoverflow.com/questions/46750420/kafka-producer-error-expiring-10-records-for-topicxxxxxx-6686-ms-has-passed) – hongsy Jan 04 '21 at 13:27

1 Answers1

1

If you have two brokers and you are able to send messages to one broker and not the other means that you might have mis-configured the other broker.

Check your advertised-listeners of the failed broker.

advertised.listeners=PLAINTEXT://1.2.3.4:9092

P.S: PLAINTEXT is a protocol and may need to be changed in your case. The IP must be accessible to your producer.

Additionally, you may also want to check the message.max.bytes on your broker or max.message.bytes on topic is greater than your max.request.size of your producer. To debug this, try sending small messages, if they also give the same error, then this is not the problem.

Reference: Stackoverflow answer

JavaTechnical
  • 8,846
  • 8
  • 61
  • 97