0

I'm new with kafka and i'm trying to do something simple.

I run kafka with docker. here is my docker-compose:

version: "3.2"
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181"

  kafka:
    image: wurstmeister/kafka
    depends_on:
      - zookeeper
    ports:
      - "9092-9094:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 127.0.0.1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  kafkaui:
    image: provectuslabs/kafka-ui
    depends_on:
      - kafka
      - zookeeper
    ports:
      - "8080:8080"
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: "kafka:9092"

As you can see, I put the My IP as 127.0.0.1 for KAFKA_ADVERTISED_HOST_NAMEE

then i do: docker-compose up -d and i check with docker ps -a and I see this screenshot enter image description here

Then I raise the number of broker : docker-compose up --scale kafka=3 -d I check with docker ps -a and I see this screeshot: enter image description here

on my browser, i go to : http://127.0.0.1:8080 and I see that I have 1 cluster offline, see screenshot: enter image description here

why??? I have an old computer: linux mint 21, 4 Gb RAM, intel Core i3

Do I need more RAM?

marietar
  • 11
  • 2

1 Answers1

0

RAM is not the problem

It's offline because it cannot connect. KAFKA_ADVERTISED_HOST_NAME: 127.0.0.1 + ports: 9092:9092 means it only accept connections from the external host, not other containers. You will not be able to scale the container with that variable set, as it needs to be kafka, but it also cannot be hardcoded to kafka when you scale beyond one broker (in other words, each container needs a unique hostname). This page covers more details. But, one solution is using KAFKA_LISTENERS: PLAINTEXT://:9092 instead, and remove ports from Compose unless you really need connectivity outside of Docker.

There is no real benefit to running more than one broker on the same host, anyway.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • For docker-compose, i changed the kafka part by removing port and adding the new environment variable: ` kafka: image: wurstmeister/kafka depends_on: - zookeeper environment: KAFKA_ADVERTISED_HOST_NAME: 127.0.0.1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_LISTENERS: PLAINTEXT://:9092 volumes: - /var/run/docker.sock:/var/run/docker.sock ` In the brower, for a few seconds i can see the cluster is online but then it is offline again. later i want to run a python script where i simulate production of topics: need outside connection? – marietar Feb 02 '23 at 20:02
  • If you only have one broker, [then you can use a port forward](https://stackoverflow.com/questions/51630260/connect-to-kafka-running-in-docker). I don't use that UI tool, so I cannot help there. – OneCricketeer Feb 02 '23 at 20:38
  • sorry I don't know how to use markdown to pretify my docker compose – marietar Feb 02 '23 at 20:49
  • Within a comment, you can't – OneCricketeer Feb 02 '23 at 20:50