Accessing Kafka broker in Docker container

Question

I can't for the life of me find information on this so forgive me if this is obvious.

I am publishing a Kafka broker's port to my host machine (MacOS) and cannot connect from the host. Then, even when I exec into the container I can't ping localhost:9092 or curl it.

I've also tried hitting the KAFKA_ADVERTISED_LISTENERS value which is PLAINTEXT://kafka1:9092 and that gives me a ping: cannot resolve PLAINTEXT://kafka1:9092

This is the relevant bit of my docker-compose file:

version: '3'
services:
  zookeeper:
    container_name: zookeeper
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      KAFKA_OPTS: "-Dzookeeper.4lw.commands.whitelist=*"
    ports:
      - "2181:2181"

  kafka1:
    build: .
    container_name: "kafka1"
    depends_on:
      - zookeeper
    ports:
      - "8778"
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OPTS: '-javaagent:/usr/jolokia/agents/jolokia-jvm.jar=host=0.0.0.0'

Dockerfile is as follows:

FROM confluentinc/cp-kafka:latest

ENV JOLOKIA_VERSION 1.3.5
ENV JOLOKIA_HOME /usr/jolokia-${JOLOKIA_VERSION}
RUN curl -sL --retry 3 \
  "https://github.com/rhuss/jolokia/releases/download/v${JOLOKIA_VERSION}/jolokia-${JOLOKIA_VERSION}-bin.tar.gz" \
  | gunzip \
  | tar -x -C /usr/ \
 && ln -s $JOLOKIA_HOME /usr/jolokia \
 && rm -rf $JOLOKIA_HOME/client \
 && rm -rf $JOLOKIA_HOME/reference

Any ideas?

score 0 · Answer 1 · answered Aug 25 '20 at 07:46

If you try to subscribe to topics from inside the container, you should specify bootstrap server as kafka1:9092. This setup works for me fine and I'm able to curl it from local machine via

curl localhost:9092
curl: (52) Empty reply from server

Additionally, you can try to add some more env variables

kafka1:
    environment:
    - BOOTSTRAP_SERVERS=kafka2:9092
    - ZOOKEEPER=zookeeper:2181

Accessing Kafka broker in Docker container

1 Answers1