0

I have a docker compose file like this:

  zookeeper:
    image: 'arm64v8/zookeeper:latest'
    container_name: zookeeper
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes

  kafka:
    image: 'wurstmeister/kafka:2.13-2.8.1'
    container_name: kafka
    ports:
      - 29092:9092 # later I also try 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

      KAFKA_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: false
      KAFKA_CREATE_TOPICS: sometopic:1:1
    depends_on:
      - zookeeper

  kafka-ui:
    container_name: kafka-ui
    image: consdata/kouncil:latest
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      bootstrapServers: kafka:9092
    depends_on:
      - zookeeper
      - kafka
    ports:
      - "8083:8080"

The kafka-ui application within Docker can successfully connect to Kafka. However when an application outside Docker tries to publish a message to Kafka at localhost:29092, I see this error:

kafka.(*Client).Produce: dial tcp: lookup kafka: no such host

It seems the host machine process itself is trying to reach hostname kafka, not a process within the Docker container. Why is that?

I also tried changing the port mapping of the kafka container from 29092:9092 to 29092:29092 but that doesn't work either (fails even sooner):

unexpected EOF

How do I fix my configuration to make Kafka reachable from the host machine?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
BeetleJuice
  • 39,516
  • 19
  • 105
  • 165
  • 1
    Ports `29092:29092` is correct. But you must set `KAFKA_LISTENERS` to include `PLAINTEXT_HOST://0.0.0.0:29092`. Do not change your advertised listeners – OneCricketeer Oct 27 '22 at 15:41
  • @OneCricketeer thank you; that fixed it :-). Do you understand why the error was an EOF when I set the port binding to 29092:29092 before? I would have expected a "can't connect" or connection refused" or something. – BeetleJuice Oct 27 '22 at 18:12
  • 1
    By setting the listener to use localhost, it'll not accept external connections. end-of-file (EOF) is networking way of saying connection established, but communication not accepted – OneCricketeer Oct 28 '22 at 14:09

0 Answers0