1

everybody,there is a virtual server in the local area network which ip is 192.168.18.230, and my machine ip is 192.168.0.175. Today, I try to use my machine (192.168.0.175) to send some messages to my virtual server(192.168.18.230), with the Kafka console producer 

$ bin/kafka-console-producer.sh --broker-list 192.168.18.230:9092 --topic test

but there is something wrong. The description of the problem is :

[2017-04-10 17:25:40,396] ERROR Error when sending message to topic test with key: null, value: 6 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0 due to 1568 ms has passed since batch creation plus linger time

But when I use the kafka-topics script to list topics, it works:

$ bin/kafka-topics.sh --list --zookeeper 192.168.18.230:2181

This problem confused me a very long period, can any body help me to solve it?

miguno
  • 14,498
  • 3
  • 47
  • 63
T.Arthur
  • 135
  • 13
  • First, can you actually reach the broker's port `192.168.18.230:9092` from your machine at `192.168.0.175`? Your `netstat -nlp | grep 9092` command was run on your VM, so it doesn't help to answer my first question. Could you share the netstat output? Also, can you share your broker's config aka `server.properties`? I suppose your problem is similar to https://stackoverflow.com/questions/38260091/kafka-0-10-java-client-timeoutexception-batch-containing-1-records-expired and/or https://stackoverflow.com/questions/34794260/when-does-the-apache-kafka-client-throw-a-batch-expired-exception. – miguno Apr 12 '17 at 06:56

2 Answers2

1

If you have a zookeeper instance which is running, you can of course ask the list of topics. However, it seems that you have no Kafka broker available.

You maybe have a zookeeper running but not Kafka.

ImbaBalboa
  • 851
  • 9
  • 23
  • Thank you very much for answer , but I have used the command "netstat -nlp | grep 9092" to ensure the kafka process is running in my virtual server , and in the other way I used the kafka's command produce and consume command in the virtual server to test if it is running normally. – T.Arthur Apr 11 '17 at 00:27
  • Make sure you have opened port 9092. – Girdhar Sojitra Apr 12 '17 at 07:05
  • I have opened port 9092....I am sure... the condition is that I test the producer and consumer in my virtual machine. like one terminal input "123" and another terminal can output "123". but when I run the consumer in my virtual machine, and run the producer in own machine, when I input "123" , my virtual machine cannot output "123", and the error is just like the problem....... – T.Arthur Apr 12 '17 at 08:41
  • On which machine your Kafka broker is running ? – ImbaBalboa Apr 12 '17 at 09:06
0

Your Kafka producer might be running on a machine which cannot access your virtual machine where your Kafka broker is running.

Also, not only should the broker port be open, but also it must be answerable by the broker i.e, the (advertised) listeners of your Kafka broker must have your virtual machine IP (IP accessible from where your Kafka producer is running, because a VM can have multiple IPs and there is no rule that all IPs will be accessible).

For example, your virtual machine have two IPs 1.2.3.4 and 4.3.2.1 and your producer on another machine points to 1.2.3.4, you must be able first to ping and telnet to this IP.

Next, you must have this IP 1.2.3.4 in the advertised listeners in your Kafka broker.

advertised.listeners=PLAINTEXT://1.2.3.4:9092

You should set this IP only as your bootstrap.servers in your Kafka Producer.

You should also ensure that the port is not just open to localhost or 127.0.0.1 like for example, when you do netstat, it should not have just localhost:9092 or 127.0.0.1:9092, it should use any local address 0.0.0.0 or your IP 1.2.3.4:9092

JavaTechnical
  • 8,846
  • 8
  • 61
  • 97