1

I am trying to create a producer:

        producer = KafkaProducer(bootstrap_servers=SERVER, security_protocol="PLAINTEXT") # this line does not have issue. From my experiments, if it cannot connect to broker, it will throw error NoBrokersAvailable


        producer.send("g", "test") 

error:

decoding-server  | [E 221216 16:23:01 conn:418] Connect attempt to <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv4 ('127.0.0.1', 29092)]> returned error 111. Disconnecting.
decoding-server  | [I 221216 16:23:01 conn:919] <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv4 ('127.0.0.1', 29092)]>: Closing connection. KafkaConnectionError: 111 ECONNREFUSED
decoding-server  | [W 221216 16:23:01 client_async:331] Node 1 connection failed -- refreshing metadata
decoding-server  | [I 221216 16:23:02 conn:380] <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv6 ('::1', 29092, 0, 0)]>: connecting to localhost:29092 [('::1', 29092, 0, 0) IPv6]
decoding-server  | [E 221216 16:23:02 conn:418] Connect attempt to <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv6 ('::1', 29092, 0, 0)]> returned error 99. Disconnecting.
decoding-server  | [I 221216 16:23:02 conn:919] <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv6 ('::1', 29092, 0, 0)]>: Closing connection. KafkaConnectionError: 99 EADDRNOTAVAIL
decoding-server  | [W 221216 16:23:02 client_async:331] Node 1 connection failed -- refreshing metadata
decoding-server  | [I 221216 16:23:03 conn:380] <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv4 ('127.0.0.1', 29092)]>: connecting to localhost:29092 [('127.0.0.1', 29092) IPv4]
decoding-server  | [E 221216 16:23:03 conn:418] Connect attempt to <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv4 ('127.0.0.1', 29092)]> returned error 111. Disconnecting.
decoding-server  | [I 221216 16:23:03 conn:919] <BrokerConnection node_id=1 host=localhost:29092 <connecting> [IPv4 ('127.0.0.1', 29092)]>: Closing connection. KafkaConnectionError: 111 ECONNREFUSED
decoding-server  | [W 221216 16:23:03 client_async:331] Node 1 connection failed -- refreshing metadata

However, I do not think it's not able to connect, because the following code works:

            consumer = KafkaConsumer(
                "g",
                group_id="g",
                bootstrap_servers=SERVER,
            )

I am running the containers locally using docker-compose. docker-compose:

zookeeper:
        image: confluentinc/cp-zookeeper:latest
        environment:
            ZOOKEEPER_CLIENT_PORT: 2181
            ZOOKEEPER_TICK_TIME: 2000
        ports:
            - 22181:2181

    kafka:
        image: confluentinc/cp-kafka:5.3.1
        depends_on:
            - zookeeper
        ports:
            - 29092:29092
        environment:
            KAFKA_BROKER_ID: 1
            KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
            KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
            KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
jigiy43106
  • 21
  • 2
  • `decoding-server` is trying to connect to `localhost` (itself), when it should be trying to connect to an **external service** `kafka:9092`. Put both containers in the same compose file as well. You may also want to upgrade your kafka container version – OneCricketeer Dec 17 '22 at 20:39

0 Answers0