Bitnami Kafka docker container not starting on Ubuntu

Question

This is the docker-compose.yml file I have prepared to start the bitnami/zookeeper and the bitnami/kafka containers.

version: "3.4"

services:
  zookeeper:
    image: bitnami/zookeeper
    restart: always
    ports:
      - "2181:2181"
    volumes:
      - "zookeeper_data:/bitnami"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: bitnami/kafka
    ports:
      - "9092:9092"
    volumes:
      - "kafka_data:/bitnami"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
    depends_on:
      - zookeeper

volumes:
  zookeeper_data:
    driver: local
  kafka_data:
    driver: local
   
networks:
  default:
    external:
      name: techBankNet

When I run the command docker-compose up -d, it shows no errors. But when I execute docker ps, I see that only the zookeeper container is running.

After checking the logs for the bitnami/kafka container, this is the output I found.

kafka 14:50:00.34 
kafka 14:50:00.34 Welcome to the Bitnami kafka container
kafka 14:50:00.34 Subscribe to project updates by watching https://github.com/bitnami/containers
kafka 14:50:00.34 Submit issues and feature requests at https://github.com/bitnami/containers/issues
kafka 14:50:00.34 
kafka 14:50:00.34 INFO  ==> ** Starting Kafka setup **
kafka 14:50:00.37 WARN  ==> KAFKA_CFG_LISTENERS must include a listener for CONTROLLER
kafka 14:50:00.37 WARN  ==> You set the environment variable ALLOW_PLAINTEXT_LISTENER=yes. For safety reasons, do not use this flag in a production environment.
kafka 14:50:00.37 INFO  ==> Initializing Kafka...
kafka 14:50:00.38 INFO  ==> No injected configuration files found, creating default config files
kafka 14:50:00.42 INFO  ==> Initializing KRaft...
kafka 14:50:00.42 WARN  ==> KAFKA_KRAFT_CLUSTER_ID not set - If using multiple nodes then you must use the same Cluster ID for each one
kafka 14:50:01.29 INFO  ==> Generated Kafka cluster ID 'W7_NXO5cQXWLs5xG4i7zYw'
kafka 14:50:01.29 INFO  ==> Formatting storage directories to add metadata...

I cannot figure out what prevents the bitnami/kafka from starting. I have checked whether any process uses port 9092 and found none. In addition, techBankNet already exists in the docker network list. This is the output of this command, docker network list.

NETWORK ID     NAME                                 DRIVER    SCOPE
b65deecbaa75   bridge                               bridge    local
5a95427177c5   host                                 host      local
c4243c86eba1   kafka-stack-docker-compose_default   bridge    local
99063d81411a   none                                 null      local
9a1c188071fb   pss-backend-application_pss          bridge    local
5fe91218bb8e   techBankNet                          bridge    local

Can anyone give me insights on why the bitnami/kafka docker container fails to start? My OS is KUbuntu 22.04.

According to the bitnami Github repo, they know the Kafka container is broken and are "working on it". You could use Confluent as an alternative https://github.com/confluentinc/cp-all-in-one/blob/7.4.0-post/cp-all-in-one-community/docker-compose.yml — OneCricketeer, Jun 30 '23 at 20:01

score 1 · Answer 1 · answered Jun 28 '23 at 15:22

Take a look at this issue which describes a similar problem (the Bitnami kafka container exiting with an error status but no errors are printed in the log).

You can set BITNAMI_DEBUG=true in the environment configuration to see additional logging:

kafka:
  image: bitnami/kafka
  ports:
    - "9092:9092"
  volumes:
    - "kafka_data:/bitnami"
  environment:
    - BITNAMI_DEBUG=true
    - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
    - ALLOW_PLAINTEXT_LISTENER=yes
    - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
  depends_on:
    - zookeeper

If I do that and try to bring up your compose configuration, I see:

kafka_1      | kafka 15:17:38.39 INFO  ==> Formatting storage directories to add metadata...
kafka_1      | Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: controller.listener.names must contain at least one value appearing in the 'listeners' configuration when running the KRaft controller role
kafka_1      |  at scala.Predef$.require(Predef.scala:281)
kafka_1      |  at kafka.server.KafkaConfig.validateControllerListenerExistsForKRaftController$1(KafkaConfig.scala:2227)
kafka_1      |  at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:2289)
kafka_1      |  at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:2160)
kafka_1      |  at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1568)
kafka_1      |  at kafka.tools.StorageTool$.$anonfun$main$1(StorageTool.scala:50)
kafka_1      |  at scala.Option.flatMap(Option.scala:271)
kafka_1      |  at kafka.tools.StorageTool$.main(StorageTool.scala:50)
kafka_1      |  at kafka.tools.StorageTool.main(StorageTool.scala)
container_kafka_1 exited with code 1

Making that error message readable without scrolling:

Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: controller.listener.names must contain at least one value appearing in the 'listeners' configuration when running the KRaft controller role

I don't know enough about Kafka to correct the configuration, but hopefully that's enough to point you in the right direction.

Yeah, I could see that. The container was up and running when I added this line `- KAFKA_ENABLE_KRAFT=no` under the environments tab. It seems there should be at least one name in the listeners' configuration when in kraft mode. — lone wolf, Jun 28 '23 at 15:31

Bitnami Kafka docker container not starting on Ubuntu

1 Answers1