0

I've been following this guide on how to create my own apache Kafka docker-compose instance for my own learning.

On my end, I only have a few services defined in my docker-compose.yml

version: "3.8"
services:
  zookeeper-1:
    container_name: zookeeper-1
    build:
      context: ./zookeeper
    volumes:
      - ./zookeeper/zookeeper.properties:/kafka/config/zookeeper.properties
      - ./data/zookeeper-1:/tmp/zookeeper/
    networks:
      - kafka
  kafka-1:
    container_name: kafka-1
    build:
      context: ./kafka
    volumes:
      - ./kafka/server.properties:/kafka/config/server.properties
      - ./data/kafka-1:/tmp/kafka-logs/
    ports:
      - "9092:9092"
    networks:
      - kafka
networks:
  kafka:
    name: kafka

When I do a docker compose up then a docker ps get me this

CONTAINER ID   IMAGE                      COMMAND                CREATED         STATUS         PORTS                    NAMES
4aaf6e37e999   apacha-kafka_zookeeper-1   "start-zookeeper.sh"   2 minutes ago   Up 2 minutes                            zookeeper-1
1a8ca86cb2d2   apacha-kafka_kafka-1       "start-kafka.sh"       2 minutes ago   Up 2 minutes   0.0.0.0:9092->9092/tcp   kafka-1

The issue I am getting is that I cannot access kafka-1 outside of the docker local instance. I've tried to consume any messages from any topic from the host machine via <kafka-host-directory>/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic example --from-beginning it always retuns me this

[2022-04-16 03:48:01,673] WARN [Consumer clientId=console-consumer, groupId=console-consumer-8373] Error connecting to node 1a8ca86cb2d2:9092 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: 1a8ca86cb2d2
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
    ...

Do I need to add EXPOS 9092 within ./kafka/Dockerfile, but even that approach didn't do anything despite rebuilding the image and the stack.

David B
  • 3,269
  • 12
  • 48
  • 80
  • 1
    Your problem starts here - the advertised listeners default to the hostname of the container. After you fix this, only then do your Docker settings matter - https://github.com/marcel-dempers/docker-development-youtube-series/blob/master/messaging/kafka/config/kafka-1/server.properties#L36 – OneCricketeer Apr 15 '22 at 22:34
  • The only reason the github documentation (might) work is because they're using `docker exec` and running commands inside the Docker network, where the hostnames can be resolved correctly – OneCricketeer Apr 15 '22 at 22:37
  • 1
    Setting the listeners did the trick simply by setting these at `server.properties` - `advertised.listeners=PLAINTEXT://localhost:9092` and `listener.security.protocol.map=PLAINTEXT:PLAINTEXT`. Thanks @OneCricketeer – David B Apr 16 '22 at 07:59

0 Answers0