16

I have two servers on VirtualBox guests each ubuntu. I can SSH from my main machine to both, and between the two so they all have the natnetwork.

I ran on one server kafka as described here:

https://kafka.apache.org/quickstart

So I brought up singlenode zookeper Kafka then started. I added the test topic. (All on MachineA . 10.75.1.247)

I am trying to list the topics on that node from another machine:

bin/kafka-topics.sh --list --bootstrap-server 10.75.1.247:9092

from MachineB (10.75.1.2)

doing that, causes the error over and over:

[2019-09-16 23:57:07,864] WARN [AdminClient clientId=adminclient-1] Error connecting to node ubuntukafka:9092 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: ubuntukafka
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:104)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:943)
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:288)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.sendEligibleCalls(KafkaAdminClient.java:925)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1140)
    at java.base/java.lang.Thread.run(Thread.java:834)

it does resolve the name (says ubuntukafka instead of ubuntukafkanode) but fails.

What am I missing? Am I using kafka wrong? I thought I could have a nice kafka server where all my other servers with data can produce information too. Then many other consumers can read the information from?

Ultimately what I wanted to test was if I could send messages to my kafka server:

bin/kafka-console-producer.sh --broker-list 10.75.1.247:9092 --topic test

And even then use python later to produce messages to the server.

from kafka import KafkaProducer

producer = KafkaProducer(bootstrap_servers='10.75.1.247:9092')
for _ in range(100):
    try:
        producer.send('test', b'some_message_bytes')
    except:
        print('doh')
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Codejoy
  • 3,722
  • 13
  • 59
  • 99

3 Answers3

31

Generally, seems your hostnames aren't resolvable. Does ping ubuntukafka work? If not, then you'll need to adjust what you're making Kafka return via advertised.listeners to be the external IP rather than the hostname

listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://10.75.1.247:9092

Where, 10.75.1.247 is the network address to be resolved by the external machines (e.g. make sure you can ping that address, too)

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • you are right it does not respond to the ubuntukafka, that is just the machine name. It gets it somehow (its not in a hosts file anywhere unless ubuntu automatically did that) in either case, where does the advertised.listeners go in the kafka/config folder? is PLAINTEXT in the above example just 10.75.1.247 or the ip address of machine A? and the advertised.listeners is on machine A? – Codejoy Sep 17 '19 at 02:41
  • 3
    PLAINTEXT is a protocol. Both lines already exist as part of `server.properties`, they just need to be adjusted and uncommented. https://stackoverflow.com/questions/42998859/kafka-server-configuration-listeners-vs-advertised-listeners – OneCricketeer Sep 17 '19 at 02:50
  • Each Kafka server requires its own unique advertised listener for clients to connect to. It could be a hostname, but it appears your DNS or hosts file isn't setup, so just use IPs – OneCricketeer Sep 17 '19 at 02:52
  • 1
    I got it working just great with this info using their .sh files to test with. Still no go on the python but that is another issue/topic to be sure. – Codejoy Sep 17 '19 at 15:40
1

only changing listeners=PLAINTEXT://localhost:9092 work for me no need to change advertised.listeners property in server config

kapil07
  • 21
  • 2
0

You can add below into file /etc/hosts:

127.0.0.1 ${your/hostname}
yunzhi
  • 131
  • 6