0

I am running the below code in docker container to create producer and to send message to particular topic but getting the error "kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.".

docker-compose.yml

  version: "3.9"
     services:
       zookeeper:
          image: 'bitnami/zookeeper:latest'
          container_name: zookeeper3
          ports:
               - '2181:2181'
          environment:
               - ALLOW_ANONYMOUS_LOGIN=yes
       kafka:
          image: 'bitnami/kafka:latest'
          container_name: kakfa3
          ports:
                - '9092:9092'
          environment:
                - KAFKA_BROKER_ID=1
                - KAFKA_LISTENERS=PLAINTEXT://:9092
                - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
                - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
                - ALLOW_PLAINTEXT_LISTENER=yes
          depends_on:
                - zookeeper
       python:
           container_name: python3
           build: .
           ports:
               - '5000:5000'
           depends_on:
               - kafka

Python code:

         from kafka import KafkaProducer
         import csv
         import json

         #Creating producer
         producer = KafkaProducer(bootstrap_servers=['localhost:9092'],value_serializer=lambda m: json.dumps(m).encode('utf-8'))

         #Declaring variable to store json message
         jsonArray =[]    

         #Reading csv file and append data as json
         with open('sample.csv', 'r') as file:
            csvReader = csv.DictReader(file)
            for row in csvReader: 
                     #add this python dict to json array
                     jsonArray.append(row)
        
         #Send message to topic one by one    
         for i in range(len(jsonArray)):
             producer.send('part1',jsonArray[i])
             producer.flush()
bharani
  • 1
  • 1
  • 1
    you trying to connect `'localhost:9092'`. It is correct only if kafka server running inside the same container (I guess don't). – rzlvmp Aug 31 '21 at 05:30
  • I changed it as 'kafka:9092' since i mentioned the port number as 9092 for kafka in docker-compose file. Even though i'm getting the same error – bharani Aug 31 '21 at 09:29
  • please add docker-compose yaml files of kafka and producer (by the way the problem in question must be reproducible) – rzlvmp Aug 31 '21 at 09:33
  • Added the docker-compose.yml file – bharani Aug 31 '21 at 09:48
  • 1
    Not sure, but `KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092` looking not correct. Did you try `KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092` and `KAFKA_LISTENERS=PLAINTEXT://kafka:9092`? – rzlvmp Aug 31 '21 at 10:00
  • Please read the documentation for the bitnami Kafka container. You're using the wrong variables – OneCricketeer Aug 31 '21 at 12:27

0 Answers0