0

I want to run kafka and debezium in docker. I write to try docker-compose file but i'm getting this error 'Failed to resolve 'kafka:9092': nodename nor servname provided, or not known '. In my code base Bootstrap server config => "BootstrapServer":"kafka:9092" enter image description here

My Docker compose file

  version: "3.4"
  services:
    postgres:
      container_name: postgres
      image: postgres
      environment:
        POSTGRES_USER: ${POSTGRES_USER:-fatih}
        POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-admin}
        PGDATA: /data/postgres
      command:
        - "postgres"
        - "-c"
        - "wal_level=logical"
      volumes:
        - ./postgres-data:/var/lib/postgresql/data
      ports:
        - "5431:5432"
      restart: unless-stopped
      
    pgadmin:
      container_name: pgadmin
      image: dpage/pgadmin4
      environment:
        PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL:-fatih@fatih.org}
        PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD:-admin}
        PGADMIN_CONFIG_SERVER_MODE: 'False'
      ports:
        - "${PGADMIN_PORT:-5050}:80"
      depends_on:
        - postgres
        
    zookeeper:
      image: confluentinc/cp-zookeeper
      restart: always
      ports:
        - "2181:2181"
      volumes:
        - "zookeeper_data:/confluent"
      environment:
        - ALLOW_ANONYMOUS_LOGIN=yes
        - ZOOKEEPER_CLIENT_PORT=2181
    kafka:
      container_name: kafka
      image: confluentinc/cp-kafka:latest
      ports:
        - "9092:9092"
      restart: always
      volumes:
        - "kafka_data:/confluent"
      environment:
        - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
        - ALLOW_PLAINTEXT_LISTENER=yes
        - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
        - KAFKA_LOG_CLEANER_DELETE_RETENTION_MS=5000
        - KAFKA_BROKER_ID=1
        - KAFKA_MIN_INSYNC_REPLICAS=1
        - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
      depends_on:
        - zookeeper
        - postgres
    connector:
      image: debezium/connect:latest
      ports:
        - "8083:8083"
      environment:
        - BOOTSTRAP_SERVERS=kafka:9092
        - GROUP_ID=1
        - CONFIG_STORAGE_TOPIC=my_connect_configs
        - OFFSET_STORAGE_TOPIC=my_connect_offsets
        - STATUS_STORAGE_TOPIC=my_connect_statuses
      depends_on:
        - zookeeper
        - kafka
  
  volumes:
    zookeeper_data:
      driver: local
    kafka_data:
      driver: local

My Docker container status enter image description here

Actually seems everything is ok. Why I getting this Error?

Fatih
  • 91
  • 1
  • 9
  • What do you mean by "your code base"? Your error is not coming from Debezium. Please read the duplicate post again about connecting to the container from **outside a container** – OneCricketeer Mar 07 '23 at 16:21
  • 1
    My codebase is mean my bootstrap servers config -> ConsumerConfig { BootstrapServers = "kafka:9092", GroupId = "gid-consumers", AutoOffsetReset = AutoOffsetReset.Earliest }; – Fatih Mar 07 '23 at 16:28
  • Im gonna exam the duplicate post thanks. – Fatih Mar 07 '23 at 16:30
  • Sure. This has nothing to do with Debezium or Postgres, so I suggest removing that from the question and showing your actual code. – OneCricketeer Mar 07 '23 at 16:31

1 Answers1

0

Try to update kafka environment by adding KAFKA_LISTENER_SECURITY_PROTOCOL_MAP and KAFKA_INTER_BROKER_LISTENER_NAME so it will look like this:

kafka:
  container_name: kafka
  image: confluentinc/cp-kafka:latest
  ports:
    - "9092:9092"
  restart: always
  volumes:
    - "kafka_data:/confluent"
  environment:
    - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
    - ALLOW_PLAINTEXT_LISTENER=yes
    - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
    - KAFKA_LOG_CLEANER_DELETE_RETENTION_MS=5000
    - KAFKA_BROKER_ID=1
    - KAFKA_MIN_INSYNC_REPLICAS=1
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
    - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT
    - KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
  depends_on:
    - zookeeper
    - postgres
artiomi
  • 383
  • 2
  • 7
  • i tried it. Error '1:1 brokers down' gone but Error '%3|1678204656.489|ERROR|rdkafka#producer-1| [thrd:app]: rdkafka#producer-1: kafka:9092/1: Failed to resolve 'kafka:9092': nodename nor servname provided, or not known (after 2ms in state CONNECT, 1 identical error(s) suppressed) ' going on . – Fatih Mar 07 '23 at 15:58
  • I still dont connect to kafka. – Fatih Mar 07 '23 at 16:00
  • `PLAINTEXT:PLAINTEXT` should be the default map. And there is only one broker, so inter-broker settings aren't needed – OneCricketeer Mar 07 '23 at 16:20