0

I have an issue with kafka docker image. I am using also kafkamanager and I am running those images from .yml file. The thing is that on localhost everything works fine but on remote server it doesn't.

I am using this .yml file:

version: "2"

services:
  kafkaserver:
    image: "spotify/kafka:latest"
    container_name: kafka
    hostname: kafkaserver
    networks:
     - kafkanet
    ports:
      - 2181:2181
      - 9092:9092
    environment:
      ADVERTISED_HOST: kafkaserver
      ADVERTISED_PORT: 9092
  kafka_manager:
    image: "mzagar/kafka-manager-docker:1.3.3.4"
    container_name: kafkamanager
    networks:
      - kafkanet
    ports:
      - 9000:9000
    links:
      - kafkaserver
    environment:
      ZK_HOSTS: "kafkaserver:2181"

networks:
  kafkanet:
    driver: bridge

I am developing in python using kafka-python. WHen i start my program, program stops at the point when producer should start sending messages. But he doesn't.

After a minute program will give me this error message:

kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
kamco
  • 45
  • 6
  • 2
    https://rmoff.net/2018/08/02/kafka-listeners-explained/ – Robin Moffatt Apr 14 '20 at 16:00
  • well nice blog, now how to push it to live? for example I want to execute program from server 1.1.1.1. This program is connecting to kafka running on server 2.2.2.2. How do I set there listener? and where should I add those listeners? to .yml file?? – kamco Apr 14 '20 at 16:31
  • First of all, don't use the spotify image. – OneCricketeer Apr 15 '20 at 00:45
  • Problem - `bootstrap.servers` to `localhost:9092` is going to *return* (see link above) the address of `kafkaserver:9092`... You must containerize your Python code, or you need `ADVERTISED_HOST=localhost`., But you should really be using `ADVERTISED_LISTENERS` because the host and port properties are deprecated (and that Spotify image is not maintained) – OneCricketeer Apr 15 '20 at 00:47
  • I tried this one. I am pointing to this https://github.com/confluentinc/examples/blob/5.3.1-post/cp-all-in-one/docker-compose.yml. I used this in my .yml file -> KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://x.y.z.x:9092 .. Of course with the IP instead of characters. I tried to run my code from this server I add to listener and still no working. – kamco Apr 15 '20 at 10:09
  • note: docker is running on x.x.x.x server and my code is running from y.y.y.y server – kamco Apr 15 '20 at 11:52
  • Why don't you just run your code in a Docker container on x.x.x.x? – OneCricketeer Apr 15 '20 at 17:46
  • cause it is part of my diploma thesis and i must run it on y.y.y.y server – kamco Apr 15 '20 at 23:22
  • Finally I set listeners in right way and it works! thanks guys – kamco Apr 16 '20 at 09:14

0 Answers0