17

I have configured kafka_2.11-2.3.0 and apache-zookeeper-3.5.5-bin on Windows 10. But while running the topic creation command I am getting the below error:

C:\kafka_2.11-2.3.0>.\bin\windows\kafka-topics.bat --create --bootstrap-server 127.0.0.1:2181 --partitions 1 --replication-factor 1 --topic testD1
Error while executing topic command : org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
[2019-10-14 16:42:40,603] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
        at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
        at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
        at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:178)
        at kafka.admin.TopicCommand$TopicService$class.createTopic(TopicCommand.scala:149)
        at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:172)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

Read somewhere in stackoverflow to add listeners=PLAINTEXT://127.0.0.1:9092 in the server.properties file but that didn't work out as expected.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
P.S.Rawat
  • 259
  • 1
  • 2
  • 10
  • 2
    `--bootstrap-server 127.0.01:2181` seems to be wrong -- 1) localhost address is `127.0.0.1`, and 2) it should be kafka server not ZK, so the port must be 9092 (default). – mazaneicha Oct 14 '19 at 13:36
  • 2
    @mazaneicha: No it should be zookeeper as it manages the Kafka instance and replacing bootstrap-server with zookeeper worked. – P.S.Rawat Oct 20 '19 at 07:31
  • 2
    @p-s-rawat That doesnt sound right as the `--bootstrap-server` option was added since version 2.2.0, https://kafka.apache.org/documentation/#upgrade_220_notable – mazaneicha Oct 21 '19 at 12:03

8 Answers8

32

Zookeeper runs on 2181, not Kafka (the bootstrap server)

By default, Kafka runs on port 9092 as below

kafka-topics --bootstrap-server 127.0.0.1:9092 --topic first_topic --create --partitions 3 --replication-factor 1
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Anand Krish
  • 431
  • 4
  • 4
8

I've struggled with the same issue on linux. The recommended way to create topics is still via the broker, you shouldn't need to connect directly to zookeeper.

It turned out to be that the shell scripts need a little more configuration when connecting to a TLS endpoint:

  1. Copy the certs linked to by your jdk to a temporary location:
cp /usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts /tmp/kafka.client.truststore.jks
  1. Make a properties file (e.g. client.properties)
security.protocol=SSL
ssl.truststore.location=/tmp/kafka.client.truststore.jks

Then try running the script again, while passing the option --command-config with your properties file e.g.:

./kafka-topics.sh --bootstrap-server <server>:<port> --list --command-config client.properties

Note that the option is not consistent between the different scripts, for the console consumer / producer you'll need: --consumer.config and --producer.config

EddKing
  • 81
  • 1
  • 2
  • When I did that, the error changed and now I get `WARN [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-docker_kafka_1/172.18.0.2:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)`. I'm using a docker container, BTW. – alx - recommends codidact Aug 24 '21 at 06:51
  • @P.S.Rawat 's answer worked in this case of mine, but I have to run another script that doesn't work with the zookeeper. – alx - recommends codidact Aug 24 '21 at 06:53
  • The actual question is using the wrong port. Has nothing to do with ssl – OneCricketeer Oct 23 '21 at 13:27
3

enter image description here Replacing bootstrap-server with zookeeper fixed the issue.

P.S.Rawat
  • 259
  • 1
  • 2
  • 10
  • 1
    Since Kafka 2.2.0, the preferred approach is to use `--bootstrap-server` (see https://issues.apache.org/jira/browse/KAFKA-7433) because it eliminates the need for direct connection to ZK. So this answer is not really the answer. Maybe you installed 2.3 client but your Kafka _server_ is still < 2.2.0 ? – mazaneicha Oct 21 '19 at 12:08
  • 6
    I followed the quickstart guide (https://kafka.apache.org/quickstart) with version 2.5. It still didn't work with --bootstrap-server. It works with --zookeeper localhost:2181. What is the reason that it doesn't work with --bootstrap-server ? – Sujoy Jul 02 '20 at 03:26
  • 1
    I too tried the new version but still faced the same issue, but using zookeeper localhost:2181 works. – Vignesh Jul 14 '20 at 12:00
  • 1
    you're specifying port for zookeeper, that's why it didn't work. Should have specified port for Kafka-server, which in default case would have been 9092. – shivam13juna Jun 23 '21 at 10:02
  • 2
    Please don't use this. Zookeeper is a deprecated flag and will soon be completely removed – OneCricketeer Oct 23 '21 at 13:28
2

For version 2.* you have to create the topic using zookeper with the default port 2181 as a parameter.

For the version 3.* the zookeeper is not any more a parameter, you should use --bootstrap-server using localhost or the IP adresse of the server and the default port 9092.

Documentation

Tarik Majid
  • 23
  • 2
0

Check your broker after you get this error, Kafka broker will give you the correct IP Address in the console which is running in a different terminal.

In my case I replaced 127.0.0.1:2181 with 192.168.0.21:9092 and I was able to create a new topic successfully.

Note: use bootstrap instead of zookeeper.

Fahad Anwar
  • 1
0

For mac, It worked for me as follow when I use the bootstrap server with the kafka server port. Initially, it failed when I tried with a zookeeper.

bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic firsttopic --create --partitions 3 --replication-factor 1
CodeChanger
  • 7,953
  • 5
  • 49
  • 80
Kavitha Shanmugam
  • 1
0

I faced the same issue, but everything was working fine before.

So I changed the data directory in both the properties files: zookeeper.properties and server.properties and it started working fine again. It could be because I didn't shut down the broker and zookeeper properly before.

Kedaar Rao
  • 127
  • 1
  • 6
0

It is very simple:

Follow step 3 in this quickstart guide:

kafka-topics.bat --create --topic quickstart-events --bootstrap-server localhost:9092

https://kafka.apache.org/quickstart

For Windows use .bat instead of .sh

chirag soni
  • 936
  • 10
  • 20