I started a kafka service using docker compose:
version: "2"
services:
zookeeper:
image: 'bitnami/zookeeper:latest'
container_name: zookeeper
ports:
- 2181:2181
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'bitnami/kafka:2.4.1'
container_name: kafka
ports:
- 29092:9092
environment:
- KAFKA_BROKER_ID=1
- KAFKA_CFG_LISTENERS=INTERNAL://kafka:9092,EXTERNAL_SAME_HOST://localhost:29092
- KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,EXTERNAL_SAME_HOST://localhost:29092
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT
- KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=100
depends_on:
- zookeeper
Kafka can be started properly and I could access kafka from other service containers defined in the same docker-compose.yml
However when I tried to connect the kafka from an offsetexplorer outside of the docker compose. I've got an error that the broker defined as localhost:29092 is not reachable. zookeeper host is ok.
What I'm trying to accomplish is:
Here's the dilemma, port 9092 is used for other services to access kafka (broker specified as kafka:9092). I need another advertised listener to allow access from e.g. local host instead of services. In this case, I need a different port, I was trying to map the host port 29092 to container port 9092 but it seems like it does not work.
I've found a solution, On the KAFKA_CFG_LISTENERS line: 0.0.0.0 is the key to make it work, localhost or 127.0.0.1 does not:
Kafka access inside and outside docker Follow the above question, I changed my docker-compose.yml file as following and now I can access kafka in side container from offset explorer running on local host outside of the docker compose network.
kafka:
image: 'bitnami/kafka:2.4.1'
container_name: kafka
ports:
- '29092:29092'
environment:
- KAFKA_BROKER_ID=1
- KAFKA_CFG_LISTENERS=INTERNAL://0.0.0.0:9092,OUTSIDE://0.0.0.0:29092
- KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,OUTSIDE://localhost:29092
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,OUTSIDE:PLAINTEXT
- KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
depends_on:
- zookeeper