0

I have a container in the same docker-compose that is trying to connect to my broker. It suddenly stopped connecting and is resulting with error:

2023-02-22 14:24:00 ERROR:kafka.conn:Connect attempt to <BrokerConnection node_id=bootstrap-0 host=0.0.0.0:29092 <connecting> [IPv4 ('0.0.0.0', 29092)]> returned error 111. Disconnecting.

I have already solved this issue by adding advertised listeners, and it worked for some time. I tried changing names, adding custom links names to the container, depends_on , but nothing helps.

docker ps shows that the advertised ports are not being added in mapping.

CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS          PORTS                                              NAMES
b1892844f99a   confluentinc/cp-kafka:7.3.0       "/etc/confluent/dock…"   26 seconds ago   Up 24 seconds   0.0.0.0:9092->9092/tcp, 0.0.0.0:29092->29092/tcp   broker

Here is my compose

  zookeeper:
      image: confluentinc/cp-zookeeper:7.3.0
      hostname: zookeeper
      container_name: zookeeper
      ports:
        - "2181:2181"
      environment:
        ZOOKEEPER_CLIENT_PORT: 2181
        ZOOKEEPER_TICK_TIME: 2000

  broker:
      image: confluentinc/cp-kafka:7.3.0
      hostname: broker
      container_name: broker
      depends_on:
        - zookeeper
      ports:
        - "29092:29092"
        - "9092:9092"
      environment:
        KAFKA_BROKER_ID: 1
        KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
        KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
        KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092,PLAINTEXT_HOST://localhost:29092
        KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
        KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
        KAFKA_TOOLS_LOG4J_LOGLEVEL: ERROR

  my_container:
    image: my_container
    container_name: my_container
    ports:
      - "8000:8000"

I am connecting to the broker with my python script, using kafka-python library with a producer like here. I tried changing the bootstrap to "broker:9092", "broker:29092", "localhost:9092", "localhost:29092", nothing connects.

producer = KafkaProducer(bootstrap_servers="0.0.0.0:29092", 
                                value_serializer=lambda user: json.dumps(event).encode('utf-8'),
                                api_version=(0,11,5),
                                retries = 3,
                                batch_size = 161879)

Please help, any info or advice is more then welcome, I am very lost and my soul is decomposing itself.

EDIT: this solution doesn't help, returns exactly the same error.

LauraAlice
  • 33
  • 6
  • Try changing PLAINTEXT_HOST://localhost:29092 to PLAINTEXT_HOST://broker:29092 – MsieurKris Feb 22 '23 at 19:56
  • @KristofDcu That is fine, actually. It's the Python code that needs to use `bootstrap_servers=broker:9092`. (0.0.0.0 is not a routable IP for client) – OneCricketeer Feb 23 '23 at 00:03
  • And that **should** connect. It'll definitely have a different error, so please post what happens when you do that, instead... For example, `user` is never used in your lambda, and `event` is never defined. This has nothing to do with the Docker setup. – OneCricketeer Feb 23 '23 at 00:05
  • I made a typo while copying, the `user` in lambda is used twice, there is no `event` , so it looks like this `value_serializer=lambda user: json.dumps(user).encode('utf-8')`. Changing to `broker:9092` results in the same error – LauraAlice Feb 27 '23 at 15:18

0 Answers0