3

my problem is i have setup 2 kafka brokers and 2 zookeeper nodes on different docker containers(Ubuntu).

Here is my configuration file of server1.properties

broker.id=1

############################# Socket Server Settings 

listeners=PLAINTEXT://0.0.0.0:9092

advertised.listeners=PLAINTEXT://ipaddress_server1:9092

zookeeper.properties configuration file -

dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=200
tickTime=2000
initLimit=20
syncLimit=10

this is are the properties for both the kafka and zookeeper servers. I have given unique broker id for each server and also create myid file insite /tmp/zookeeper dir.

now when i am testing kafka cluster by producing messages through only one ip address like this ./bin/kafka-console-producer.sh --broker-list 172.171.0.3:9092 --topic demo it works fine. But when i shutdown one container which is leader, still i am getting messages from the topic. But when i am running the consumer script again it is showing me WARN msgs : -

Connection to node -1 (/172.171.0.3:9092) could not be established. Broker may not be available

Now i am not able to get the messages, what should i do ???

Nishu Tayal
  • 20,106
  • 8
  • 49
  • 101
Suraj
  • 75
  • 2
  • 13
  • Can you include the console-consumer command too? – Mickael Maison Dec 14 '18 at 14:53
  • Why are you manually using property files for containers? The Confluent, Bitnami, wurstmeister Kafka images all are completely configured through environment variables... But it any case, please show that Kafka is indeed starting rather than show the error of the clients – OneCricketeer Dec 14 '18 at 15:15
  • Possible duplicate of [Kafka Consumer does not receive data when one of the brokers is down](https://stackoverflow.com/questions/53771673/kafka-consumer-does-not-receive-data-when-one-of-the-brokers-is-down) – OneCricketeer Dec 14 '18 at 15:19
  • Hey mickael, here is the console consumer command. ./bin/kafka-console-consumer.sh — bootstrap-server 172.171.0.3:9092 —topic demo —from-beginning —partition 0 – Suraj Dec 14 '18 at 16:05
  • What is the replication factor of the consumer offsets topic? Value of property offsets.topic.replication.factor=1 in server.properties file – Sammy Dec 14 '18 at 16:38
  • Hey Sammy i have entered value 2 foe offsets.topic.replication.factor – Suraj Dec 14 '18 at 16:57

0 Answers0