I have a kafka cluster running in docker which I create with the .yml file below:
version: '2'
services:
zookeeper:
image: "confluentinc/cp-zookeeper:5.4.0"
hostname: zookeeper
ports:
- '32181:32181'
environment:
ZOOKEEPER_CLIENT_PORT: 32181
ZOOKEEPER_TICK_TIME: 2000
extra_hosts:
- "moby:127.0.0.1"
kafka:
image: "confluentinc/cp-enterprise-kafka:5.4.0"
hostname: kafka
ports:
- '9092:9092'
- '29092:29092'
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: kafka:29092
CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: zookeeper:32181
CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
CONFLUENT_METRICS_ENABLE: 'false'
CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'
extra_hosts:
- "moby:127.0.0.1"
schema-registry:
image: "confluentinc/cp-schema-registry:latest"
hostname: schema-registry
depends_on:
- zookeeper
- kafka
ports:
- '8081:8081'
environment:
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:32181
extra_hosts:
- "moby:127.0.0.1"
kafdrop:
image: "obsidiandynamics/kafdrop"
ports:
- '9000:9000'
environment:
KAFKA_BROKERCONNECT: kafka:29092
JVM_OPTS: "-Xms32M -Xmx64M"
SERVER_SERVLET_CONTEXTPATH: "/"
depends_on:
- kafka
I then have a simple producer not in Docker just on my machine that I am trying to produce messages to the cluster. The producer file is:
from kafka import KafkaProducer
import time
topicName = 'myTestTopic'
producer = KafkaProducer(bootstrap_servers=['localhost:9092', 'kafka:29092'])
while True:
message = producer.send(topicName, value=b'Hello World')
metadata = message.get()
print(metadata.topic)
print(metadata.partition)
time.sleep(5)
When I run the file and the cluster is running on docker all I get is this error
raise Errors.NoBrokersAvailable()
kafka.errors.NoBrokersAvailable: NoBrokersAvailable
Any help to resolve this issue would be great