0

I have a dockerized application that uses Kafka/Rails and I'm having problems setting it up.

My main problem at this moment is that the application seems to be connecting to the wrong port when I try to send a message on my local terminal.

When I try to send a message to my topic, the following error is returned: Error msg

My relevant env vars:

KAFKA_CONSUMER_GROUPS=kafka://172.23.0.3:9093

My container-info: Container Info

My compose file:

version: "3"

networks:
  kafka-cluster:
    name: kafka-cluster
    driver: bridge

services:
  zookeeper:
    image: bitnami/zookeeper:3.6.2
    container_name: zookeeper
    restart: always
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    networks:
      - kafka-cluster
      
  kafka1:
    image: bitnami/kafka:2.7.0
    container_name: kafka1
    restart: always
    ports:
      - '9093:9093'
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka1:9092,EXTERNAL://localhost:9093 # Set the EXTERNAL value to the IP where this docker is running
      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
      - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
    depends_on:
      - zookeeper
    networks:
      - kafka-cluster
      
  # kafka2:
  #   image: bitnami/kafka:2.7.0
  #   container_name: kafka2
  #   restart: always
  #   ports:
  #     - '9094:9094'
  #   environment:
  #     - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
  #     - ALLOW_PLAINTEXT_LISTENER=yes
  #     - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
  #     - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9094
  #     - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka2:9092,EXTERNAL://localhost:9094
  #     - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
  #     - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
  #   depends_on:
  #     - zookeeper
  #   networks:
  #     - kafka-cluster
      
  # kafka3:
  #   image: bitnami/kafka:2.7.0
  #   container_name: kafka3
  #   restart: always
  #   ports:
  #     - '9095:9095'
  #   environment:
  #     - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
  #     - ALLOW_PLAINTEXT_LISTENER=yes
  #     - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
  #     - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9095
  #     - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka3:9092,EXTERNAL://localhost:9095
  #     - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
  #     - KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
  #   depends_on:
  #     - zookeeper
  #   networks:
  #     - kafka-cluster
      
  kafdrop:
    image: obsidiandynamics/kafdrop:latest
    container_name: kafdrop
    restart: always
    ports:
      - 9001:9000
    environment:
      - KAFKA_BROKERCONNECT=kafka1:9092
      # - KAFKA_BROKERCONNECT=kafka1:9092,kafka2:9092,kafka3:9092
    depends_on: 
      - kafka1
    networks:
      - kafka-cluster

Am I doing something wrong?

Thanks in advance.

Rubens Cordeiro
  • 1
  • You are exposing your kafka1 on 9093 port. If possible i would first validate the kafka connectivity without rails app using [kafkacat](https://docs.confluent.io/platform/current/app-development/kafkacat-usage.html) or [here](https://docs.confluent.io/platform/current/tutorials/examples/clients/docs/kafka-commands.html) – teedak8s Jul 11 '22 at 19:13
  • How about running your app in its own container on the same Docker network? – OneCricketeer Jul 11 '22 at 20:13

0 Answers0