2

I am trying to connect a python kafka consumer to a kafka broker using the following script:

import kafka

from configuration import config


if __name__ == "__main__":

    print('Start')

    consumer = kafka.KafkaConsumer(
        bootstrap_servers=config.KAFKA_BOOTSTRAP_SERVERS,
        auto_offset_reset =config.CONSUMER_AUTO_OFFSET_RESET,
        enable_auto_commit=True,
        group_id=None,
    )
    consumer.subscribe((config.CONSUMER_INPUT_TOPIC,))

    print('Listening')
    for msg in consumer:
        print('Message received')

I am executing that script as an entrypoint to a custom docker image.

This script works fine when executed from my local shell with python script.py or even when I execute it manually through docker using docker-compose exec my_service python script.py but it does not run at all when executed through the entrypoint. In the latter case not even the Start is printed! One more detail is that when for msg in consumer: block is removed the script gets executed and the prints show up.

I have tested it in two pc's and received the same results.

I have tried with the following python base images:

python:3.8.5
python:3.8.13
python:3.8.13-slim-buster

The output of docker inspect <custom_image_name> | grep -A5 Cmd is:

            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "ENTRYPOINT [\"sh\" \"/entrypoint.sh\"]"
            ],
--
            "Cmd": null,
            "Image": "sha256:<sha256>",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "sh",

My Dockerfile:

FROM python:3.8.13-slim-buster

COPY ./resources/requirements.txt /requirements.txt

RUN pip install pip --upgrade \
    && pip install -r /requirements.txt

COPY ./resources/entrypoint.sh /entrypoint.sh

ENTRYPOINT ["sh", "/entrypoint.sh" ]

and the entrypoint.sh

#!/usr/bin/env bash

python run.py
Charalamm
  • 1,547
  • 1
  • 11
  • 27
  • what you are getting on running `docker inspect image |grep -A5 Cmd` – Arun Pal May 31 '22 at 09:02
  • Can you post your Dockerfile? – Vdoo May 31 '22 at 09:15
  • @arunpal Just edited to answer questions. – Charalamm May 31 '22 at 09:24
  • check if config is getting loaded properly. Also I believe you might have miss to build image again after doing changes. Or doing some typo in image building. – Arun Pal May 31 '22 at 09:34
  • 1
    Does setting an environment variable `PYTHONUNBUFFERED=1` help? Also see [Python app does not print anything when running detached in docker](https://stackoverflow.com/questions/29663459/python-app-does-not-print-anything-when-running-detached-in-docker). (If this is the problem, I'd expect you to be able to see activity from the Kafka broker end.) – David Maze May 31 '22 at 10:01
  • @DavidMaze that was it, (PYTHONUNBUFFERED=1). Could those logs block the main process? – Charalamm May 31 '22 at 11:25
  • No; if you print out enough log messages, the Python process will eventually write out a block of content. – David Maze May 31 '22 at 11:38

0 Answers0