0

I have the following file, producer.js, taken from https://kafka.js.org/docs/getting-started.

const { Kafka } = require('kafkajs')

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['broker:9092'],
})

const producer = kafka.producer()

const main = async () => {
    await producer.connect()
    await producer.send({
      topic: 'quickstart',
      messages: [
        { value: 'Hello KafkaJS user!' },
      ],
    })

    await producer.disconnect()
}

main();

I have a single Kafka broker running with Docker as defined in the docker-compose.yml.

---
version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.0.1
    platform: linux/x86_64
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-kafka:7.0.1
    platform: linux/x86_64
    container_name: broker
    ports:
    # To learn about configuring Kafka for access across networks see
    # https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/
      - "9092:9092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1

  # kafka_ui:
  #   image: provectuslabs/kafka-ui:latest
  #   # platform: linux/x86_64
  #   container_name: kafka_ui
  #   depends_on:
  #     - broker
  #   ports:
  #     - 8080:8080
  #   environment:
  #     KAFKA_CLUSTERS_0_ZOOKEEPER: zookeeper:2181
  #     KAFKA_CLUSTERS_0_NAME: local
  #     KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: broker:9092

Once I start the containers, I can even write messages to a topic that I created called quickstart. But when I try to produce through the script using node producer.js, I get the following error:

{"level":"ERROR","timestamp":"2022-07-13T09:58:42.720Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection error: getaddrinfo ENOTFOUND broker","retryCount":3,"retryTime":2142}

And of course, after a few retries, it times out. I'm not sure what I'm doing wrong as I followed the instructions from the KafkaJS documentation.

Any help would be highly appreciated!

P.S. I saw this solution and tried it but unfortunately, to no avail: Failed to connect to seed broker with kafkajs

Rahul
  • 543
  • 1
  • 7
  • 24
  • Only if you add your kafkajs service to the compose file as another service will `broker` be resolvable. And then you need to use `29092`, as defined by `KAFKA_ADVERTISED_LISTENERS`. Same answer for fixing your `kafka_ui` – OneCricketeer Jul 13 '22 at 15:42

1 Answers1

3

If your code is not running in a container, you should change broker:9092 to localhost:9092 as your advertised listener is setting localhost as the address that is to be used externally for that port.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Sarwar Bhuiyan
  • 344
  • 1
  • 7