0

I may overthinking this and I most likely am, cause I've gotten this to work before but not sure how. I have a docker container that boots up kafka and I'm trying to produce some records but getting a kafka.errors.NoBrokersAvailable: NoBrokersAvailable in my code so I'm not sure I'm calling it right. So here is a portion of my docker compose that creates the kafka instance.

  kafka:
    container_name: kafka
    image: debezium/kafka:latest
    ports:
      - "0.0.0.0:9092:9092"
    links:
      - zookeeper
    environment: 
      - ZOOKEEPER_CONNECT=zookeeper:2181
    volumes: 
        - ./kafka/data:/kafka/data
        - ./kafka/logs:/kafka/logs

And here is my python code I'm trying to run. (I also have zookeeper in my docker image above, but feeling like this piece is the piece that is relevant).

from kafka import KafkaConsumer, KafkaProducer
import json
from json import loads
from csv import DictReader 


bootstrap_servers = ['kafka:9092']
topicname='test-topic'
producer=KafkaProducer(bootstrap_servers=bootstrap_servers)
producer=KafkaProducer()

with open('test_data_producer.csv', 'r') as new_obj:
   csv_dict_reader=DictReader(new_obj)
   for row in csv_dict_reader:
       ack = producer.send(topicname, json.dumps(row).encode('utf-8'))
       metadata = ack.get()
       print(metadata.topic, metadata.partition)
Alex
  • 211
  • 1
  • 11
  • 1
    1) Show a [mcve] of your full compose file 2) You need to add Python app to the same compose file to resolve `kafka` service name. 3) Debezium image doesn't expose `ADVERTISED_HOSTNAME=kafka` by default, I don't think – OneCricketeer Dec 22 '22 at 21:04

1 Answers1

1

producer is not able to connect broker since kafka is not valid dns name, if you add following in your hosts file 127.0.0.1 kafka and 127.0.0.1 zookeeper then your producer should be able to connect kafka.

Niranjan
  • 137
  • 3