-1

I have two containers one is kafka container open port 9092:9092 and another is Fastapi container. If I don't dockerize Fastapi, I can make rest api request to fastapi to kafka. It sends message to kafka via fastapi. But when I dockerize fastapi can't connect fastapi container to kafka container. I cant run fastapi docker file with -p 8000:8000 -p 9092:9092 it says 9092 is already used.

How can I make request to fastapi container then fastapi connects to kafka container.

fastapi dockerfile

FROM python:3.8.10

ADD . .

COPY requirements.txt .

RUN pip3 install -r requirements.txt

CMD ["python3", "main.py"]

My error is

kafka.errors.NoBrokersAvailable: NoBrokersAvailable.

I get kafka container IP address and I am making to kafka container IP address example

producer = KafkaProducer(bootstrap_servers=containerip, value_serializer=lambda x: json.dumps(x).encode('utf-8'),api_version=(2)), lines=True, orient='records')
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
newUser
  • 386
  • 5
  • 17
  • Because you've shown no error or your actual Python code. For example, Kafka is no longer available at localhost:9092 if you containerize your api – OneCricketeer Apr 05 '22 at 13:51

2 Answers2

0

cant run fastapi docker file with -p 8000:8000 -p 9092:9092 it says 9092 is already used.

Remove it then. Unclear why you need port 9092 on your API, anyway ; it's not the Kafka service.

Without seeing your complete Kafka client code, it's hard to say what your other problems are, so please consult Connect to Kafka running in Docker

For example, what is containerip? This should be kafka:9092 (if you follow the instructions in the linked post)

  1. Run docker network create
  2. Make sure you use docker run with --network on both containers
  3. Ensure KAFKA_ADVERTISTED_LISTENERS variable contains at least INTERNAL://kafka:9092
  4. Remove the -p flags for the Kafka container since you are only interacting with Kafka from another container.
  5. Connect Python to kafka:9092
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • I am running fastapi container with docker run -p 8000:8000 it is working fine . Also kafka container is running opened 9092:9092 but When I make request fastapi to kafka it gives me this error kafka.errors.NoBrokersAvailable: NoBrokersAvailable. I get kafka container IP address and I am making to kafka container IP address example producer = KafkaProducer(bootstrap_servers=containerip, value_serializer=lambda x: json.dumps(x).encode('utf-8'),api_version=(2)), lines=True, orient='records') – newUser Apr 05 '22 at 14:51
  • You should not be using IP addresses when you are using Docker, as linked in the other answer - https://docs.docker.com/network/bridge/ Plus, between containers the `-p` flag or the numbers listed afterwards do not do anything. – OneCricketeer Apr 05 '22 at 16:21
-2

If you set network mode of your container to host, it's done.

run your fast-api (pay attention to network switch):

docker run --network host -p 8000:8000 fast-api

run kafka:

docker run -p 9092:9092 kafka

run postgres:

docker run -p 5432:5432 postgres

but it's better to use bridge-networks.

Noorbala7418
  • 42
  • 5