Cannot write to kafka, brokers are down

Question

I'm running Kafka locally. When I try to write to Kafka I get the following error:

kafkacat -b localhost:9092 -t req -T -P -l  msgs
hello
world
% ERROR: Local: Broker transport failure: localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused (after 1ms in state CONNECT)
% ERROR: Local: All broker connections are down: 1/1 brokers are down : terminating

I have Kafka listening on port 9092:

bash-3.2$ netstat -an | grep 9092
tcp4       0      0  127.0.0.1.9092         127.0.0.1.50994        ESTABLISHED
tcp4       0      0  127.0.0.1.50994        127.0.0.1.9092         ESTABLISHED
tcp4       0      0  127.0.0.1.9092         127.0.0.1.50986        ESTABLISHED
tcp4       0      0  127.0.0.1.50986        127.0.0.1.9092         ESTABLISHED
tcp4       0      0  127.0.0.1.9092         127.0.0.1.50984        ESTABLISHED
tcp4       0      0  127.0.0.1.50984        127.0.0.1.9092         ESTABLISHED
tcp4       0      0  127.0.0.1.9092         *.*                    LISTEN
bash-3.2$
bash-3.2$ telnet 127.0.0.1 9092
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

And here are the relevant parts of the kafka config:

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0

# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=PLAINTEXT://127.0.0.1:9092
bootstrap.servers=127.0.0.1:9092

What I am doing wrong? Why can't I write a few entries to my local Kafka?

P.S.

When I use the scripts that Kafka comes with I can create the topic:

/usr/local/bin/kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic Hoffman02
Created topic Hoffman02.

But when I use kafkacat kafkacat -b localhost:9092 -t Hoffman04 -P, it hangs forever but still I can find the file Hoffman04 in the logs folder ls -lrt /usr/local/var/lib/kafka-logs:

drwxr-xr-x  6 jenia  admin   192 28 May 11:17 Hoffman04-0

nothing realted , but I prefer to use Kafka standalone not kafkacat — silentsudo, May 28 '20 at 15:05
`kafkacat` is an excellent diagnostics tool, I would always use it for this kind of troubleshooting. — Robin Moffatt, May 28 '20 at 15:23
@jenia what's the output of `kafkacat -b localhost:9092 -L` ? — Robin Moffatt, May 28 '20 at 15:23
@RobinMoffatt the output is a very long list of partitions: ...partition 49, leader 0, replicas: 0, isrs: 0 topic "Hoffman02" with 1 partitions: partition 0, leader 0, replicas: 0, isrs: 0 ... — Jenia Ivanov, May 28 '20 at 15:32
Can you edit your question to include the output of the above command, just the list of brokers (not the topics detail) — Robin Moffatt, May 28 '20 at 15:58

score 3 · Answer 1 · answered May 29 '20 at 07:04

Depending on your OS (OSX in particular), localhost may resolve to both ipv4 and ipv6 addresses, while your broker is only binding to the IPv4 address. You can limit the address family to IPv4 in kafkacat by adding -X broker.address.family=v4 to your kafkacat command line.

Cannot write to kafka, brokers are down

1 Answers1