0

I am installing from https://github.com/conduktor/kafka-stack-docker-compose/blob/master/zk-multiple-kafka-multiple.yml unsuccessfully.

These are my docker conatiners:

f158c8d436fb   confluentinc/cp-kafka:7.1.1       "/etc/confluent/dock…"   22 minutes ago   Up 13 minutes   9092/tcp, 0.0.0.0:9093->9093/tcp, :::9093->9093/tcp                       kafka2
9bce7c4ec4d3   confluentinc/cp-kafka:7.1.1       "/etc/confluent/dock…"   22 minutes ago   Up 13 minutes   0.0.0.0:9092->9092/tcp, :::9092->9092/tcp                                 kafka1
b92a723835ea   confluentinc/cp-kafka:7.1.1       "/etc/confluent/dock…"   22 minutes ago   Up 13 minutes   9092/tcp, 0.0.0.0:9094->9094/tcp, :::9094->9094/tcp                       kafka3
0e483aa953a3   confluentinc/cp-zookeeper:7.1.1   "/etc/confluent/dock…"   22 minutes ago   Up 13 minutes   2181/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2182->2182/tcp, :::2182->2182/tcp   zoo2
2fb1b2b8e799   confluentinc/cp-zookeeper:7.1.1   "/etc/confluent/dock…"   22 minutes ago   Up 13 minutes   2181/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2183->2183/tcp, :::2183->2183/tcp   zoo3
1ffc65f8120f   confluentinc/cp-zookeeper:7.1.1   "/etc/confluent/dock…"   22 minutes ago   Up 13 minutes   2888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp, 3888/tcp             zoo1

when I try to execute some command inside the kafka1 container: kafka-console-producer --broker-list kafka1:9092 --topic first_topic I am getting:

[2022-06-26 03:03:36,048] WARN [Consumer clientId=console-consumer, groupId=console-consumer-81981] Connection to node 2147483644 (/127.0.0.1:9094) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

if I use localhost in place of kafka1 I get the same error.

Sometimes I can send the message but when I try to consume it I get the same error.

Sometimes the brokers stop without to show any error in the log container.

Thank you.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Raul Cejas
  • 399
  • 4
  • 13
  • It's `kafka1:19092`. If the brokers crash, that's a separate problem. For example, if it's running out of memory, you don't need an equal number of Zookeepers for brokers. And running multiple brokers on one machine isn't really useful. – OneCricketeer Jun 26 '22 at 13:15
  • I read about performance problems but not error problems ( https://stackoverflow.com/questions/44110161/why-single-node-multiple-broker-in-kafka-cluster-not-preferred ). In my logs of brokers and zoos I don't have errors only info and warm. I take your advice but I would like to implement my development environment similar to production. – Raul Cejas Jun 26 '22 at 13:53
  • Your producer and consumers should work the same if they communicate with one broker or 3. But you'll need to use the correct network addresses – OneCricketeer Jun 26 '22 at 13:57
  • I am not getting a connection error, in fact I am getting a warm between internal brokers, because my command is addressed to the broker1. When I create the topic it works fine. – Raul Cejas Jun 26 '22 at 14:10
  • `127.0.0.1:9094) could not be established` is definitely a connection error,but it is logged at the WARN level because the client will attempt to reconnect to other brokers. It's failing because you need to use port 19092 for `kafka1` broker, as I said in the first comment. Have you read the duplicate post yet? Do you [understand how advertised listeners property works in Kafka](https://confluent.io/blog/kafka-listeners-explained/)? It's not fully used for topic creation, only for producers or consumers – OneCricketeer Jun 26 '22 at 17:49
  • I tried as you said with kafka1:19092 and it works fine when I have only one zoo and broker. If I have the client outside docker what is the right way to send the message ? – Raul Cejas Jun 26 '22 at 19:04
  • It should also work when you have more than one. Outside of docker is also answered in the duplicate post I linked to. [Based on this line](https://github.com/conduktor/kafka-stack-docker-compose/blob/master/zk-multiple-kafka-multiple.yml#L46), you can use `127.0.0.1:9092` or 9093 & 9094 for the other brokers – OneCricketeer Jun 27 '22 at 15:01

0 Answers0