Kafka failing to connect broker on docker

Question

I have set up a Kafka cluster with docker-compose file where i specify the brokers like this:

kafka1:
    image: confluentinc/cp-kafka:latest
    hostname: kafka1
    ports:
      - "19092:19092"
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:12181,zookeeper-2:12181,zookeeper-3:12181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:19092

Now when i try to create topic with js script and i refer to the brokers like follows(for the sake of this post i only attach code about 1 broker):

try
    {
        const kafka = new Kafka({
            clientId: 'myapp',
            brokers: ['kafka1:19092','kafka2:29092','kafka3:39092']
        })

I get this error:

{"level":"ERROR","timestamp":"2020-03-21T19:06:13.653Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka1","broker":"kafka1:19092","clientId":"myapp","stack":"Error: getaddrinfo ENOTFOUND kafka1\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}

I feel like i have exhausted internet trying to troubleshoot this. Anyone have any suggestions how to connect to the broker in the docker?

Maybe you forgot scheme part of the URL? ```brokers: ['kafka1:19092','kafka2:29092','kafka3:39092']``` — D. Richard, Mar 21 '20 at 19:42
Can you explain what you mean? I didnt quite understand. I have the brokers listed the way you have in your comment — Itzblend, Mar 21 '20 at 19:48
```I mean like: brokers: ['tcp://kafka1:19092','kafka2:29092','kafka3:39092']``` — D. Richard, Mar 21 '20 at 19:54
Probably this article may be helpful https://rmoff.net/2018/08/02/kafka-listeners-explained/ — antonku, Mar 21 '20 at 21:03
Where is your JS code running? Within the Docker network, or outside? — Robin Moffatt, Mar 21 '20 at 21:19
@Robin Moffatt Im fairly new to docker so I dont know how to run this JS code on docker. At the moment im just running it locally on my computer — Itzblend, Mar 21 '20 at 21:21

score 2 · Accepted Answer · answered Mar 22 '20 at 00:11

See Robin's blog in the comments.

Otherwise, your code will only work within a docker container because you're referring to the Docker service names, which are not resolvable by your DNS server

In other words 1) adjust the advertised listeners (see blog) to include localhost 2) use localhost in your code when it runs outside a container

Note: 3 brokers on one machine isn't improving anything

Kafka failing to connect broker on docker

1 Answers1