6

I'm trying to setup Kafka in a docker container for local development. My docker-compose.yml looks as follows:

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181"
    hostname: zookeeper
  kafka:
    image: wurstmeister/kafka
    command: [start-kafka.sh]
    ports:
      - "9092"
    hostname: kafka
    environment:
      KAFKA_CREATE_TOPICS: "UploadFile:1:1,GetFile:1:1,TrackUpload:1:1,GetEmailContent:1:1" # topic:partition:replicas
      KAFKA_ADVERTISED_HOST_NAME: kafka # docker-machine ip
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_PORT: 9092
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      - "zookeeper"

Unfortunately my node app running on my localhost (not in docker) cannot connect to it. I used the url 'kafka:9092' and even 'localhost:9092'. Nothing works. Any idea what's happening?

THpubs
  • 7,804
  • 16
  • 68
  • 143
  • Possible duplicate of [Connect to Kafka running in Docker from local machine](https://stackoverflow.com/questions/51630260/connect-to-kafka-running-in-docker-from-local-machine) – OneCricketeer Aug 19 '18 at 16:57
  • If you don't follow the above, you will need to run your app within the Docker network – OneCricketeer Aug 19 '18 at 16:59

1 Answers1

5

Expose the host port 9092 for kafka service & you should be able to connect via "localhost:9092" from the app or host machine. 

  ....
  kafka:
    image: wurstmeister/kafka
    command: [start-kafka.sh]
    ports:
      - "9092:9092"
  ....
vivekyad4v
  • 13,321
  • 4
  • 55
  • 63
  • 1) The port was already given 2) I don't think that'll work (see duplicate link above) – OneCricketeer Aug 19 '18 at 16:58
  • The app is not in docker, it's running on the host machine. He cannot connect that app to kafka container without exposing the host port or using a host network mode. I believe this should resolve his connection issue at least in context of docker, not sure about the kafka configuration or dependencies. – vivekyad4v Aug 19 '18 at 17:48
  • I understand the app is not in Docker. My point is that there is additional Kafka configuration required called the "advertised listeners". Simply exposing the port like this doesn't work – OneCricketeer Aug 19 '18 at 18:40
  • Thanks, it worked. But now I can't produce from inside a Consumer. Simply a Consumer can't produce and call another consumer. Any idea why? – THpubs Aug 20 '18 at 18:28
  • If you set the port, you can't scale anymore with "docker-compose scale kafka=3" That is why you have to use - "9092" no? If you set -'9092:9092' there will be port conflict... – DeepProblems Jan 28 '19 at 16:35