Cannot connect to kafka from ourside of its docker container

Question

This is my docker compose file:

version: "3.7"

services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    container_name: zookeeper
    ports: 
      - 2181:2181
    env_file: 
      - zookeeper.env
  
  kafka:
    image: 'bitnami/kafka:latest'
    container_name: kafka
    env_file:
      - kafka.env
    ports: 
      - 9093:9092
    depends_on:
      - zookeeper

And also these are my .env files:

kakfa.env

KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
KAFKA_CFG_LISTENERS=INTERNAL://kafka:9092,EXTERNAL://localhost:9093
KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,EXTERNAL://localhost:9093
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR=1
ALLOW_PLAINTEXT_LISTENER=yes

zookeeper.env

ZOOKEEPER_CLIENT_PORT=2181
ZOOKEEPER_TICK_TIME=2000
ALLOW_ANONYMOUS_LOGIN=yes

When I make the compose file up, everything seems to be fine and I have good telnet result form outside of the docker container, on my localhost, to 9093 port.

I write a simple producer python file as below:

import logging

from kafka import KafkaProducer

logging.basicConfig(level=logging.INFO)


class Producer():
    def __init__(self) -> None:
        self.conn = KafkaProducer(bootstrap_servers="localhost:9093")

    def produce(self):
        while True:
            try:
                self.conn.send("test", b'test')
            except KeyboardInterrupt:
                logging.info("Producer finished")
                break


if __name__ == "__main__":
    producer = Producer()
    producer.produce()

When I run my code, I get this error:

WARNING:kafka.conn:DNS lookup failed for kafka:9092, exception was [Errno -3] Temporary failure in name resolution. Is your advertised.listeners (called advertised.host.name before Kafka 9) correct and resolvable?
ERROR:kafka.conn:DNS lookup failed for kafka:9092 (AddressFamily.AF_UNSPEC)

I read also this post but I could not resolve my error and I don't know what should I do to get rid of this error.

Answer 1

You've forwarded the wrong port

9093 on the host needs to map to the localhost:9093 advertised port

Otherwise, you're connecting to 9093, which returns kafka:9092, as explained in the blog. Container hostnames cannot be resolved by the host, by default