0

Our problem is that the internal property kafka.bootstrap.servers will not be overwritten by kafka:9092. Our container still tries to access localhost:9092.

We have set in our docker-compose.yml file for the environment of each microservice to: environment: - kafka.bootstrap.servers=kafka:9092 and also the same in the properties of each MS

Is there any way to overwrite this property?

edit: important parts of our docker-compose.yml file:

services:  
zookeeper:
    image: strimzi/kafka:0.19.0-kafka-2.5.0
    command:
      ["sh", "-c", "bin/zookeeper-server-start.sh config/zookeeper.properties"]
    ports:
      - "2181:2181"
    environment:
      LOG_DIR: /tmp/logs

  kafka:
    image: strimzi/kafka:0.19.0-kafka-2.5.0
    command:
      [
        "sh",
        "-c",
        "bin/kafka-server-start.sh config/server.properties --override listeners=$${KAFKA_LISTENERS} --override advertised.listeners=$${KAFKA_ADVERTISED_LISTENERS} --override zookeeper.connect=$${KAFKA_ZOOKEEPER_CONNECT}",
      ]
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      LOG_DIR: "/tmp/logs"
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  authentication-service:
    build:
      context: implementation/authentication
      dockerfile: src/main/docker/Dockerfile.jvm
    environment:
      - quarkus.mongodb.connection-string=mongodb://mongodb:27017
      - KAFKA_BOOTSTRAP_SERVERS=kafka:9092
    container_name: authentication-service-ase
    depends_on:
      - mongodb
      - kafka

The errors we keep getting are:

cart-service-ase              |
cart-service-ase              | {
cart-service-ase              |     "timestamp": "2021-06-02 12:19:48",
cart-service-ase              |     "sequence": 378,
cart-service-ase              |     "loggerClassName": "org.apache.kafka.common.utils.LogContext$LocationAwareKafkaLogger",
cart-service-ase              |     "loggerName": "org.apache.kafka.clients.NetworkClient",
cart-service-ase              |     "level": "WARN",
cart-service-ase              |     "message": "[Consumer clientId=kafka-consumer-checkout-out, groupId=cart] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.",
cart-service-ase              |     "threadName": "vert.x-kafka-consumer-thread-1",
cart-service-ase              |     "threadId": 38,
cart-service-ase              |     "mdc": {
cart-service-ase              |     },
cart-service-ase              |     "ndc": "",
cart-service-ase              |     "hostName": "34e4de1e7647",
cart-service-ase              |     "processName": "quarkus-run.jar",
cart-service-ase              |     "processId": 1
cart-service-ase              | }
Felpower
  • 87
  • 1
  • 12
  • Please [edit] the question and include the (relevant parts of) `docker-compose.yml`. – Turing85 Jun 02 '21 at 11:09
  • instead of `kafka.bootstrap.servers`, try using `KAFKA_BOOTSTRAP_SERVERS` – Turing85 Jun 02 '21 at 11:19
  • Still getting the same error saying: message: [Consumer clientId=kafka-consumer-register-out, groupId=cart] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available – Felpower Jun 02 '21 at 11:25
  • Have you tried `KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092`? – geoand Jun 02 '21 at 11:57
  • Where would you put that? Because we already have it under kafka: environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 – Felpower Jun 02 '21 at 12:05

1 Answers1

1

You need KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 since you cannot advertise localhost to other containers in the same network. As the error says, the consumer container is trying to connect with itself, not the Kafka container.

However, you'll need to verify the override flags in the command actually set the property correctly

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245