0

I am trying to install Python dependencies using Poetry and then run a Python script in a Docker container using a cronjob. However, the python script doesn't execute as expected, and I can't figure out what tweaks to my Docker/crontab setup is causing problems. I have followed several different StackOverflow threads (like this one) on how to configure cron in a container, but none of them are consistently working in my use case. Here is the Dockerfile definition:

FROM python:3.11

RUN apt-get update && apt-get -y install cron
# Install Poetry
RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python3 && \
    cd /usr/local/bin && \
    ln -s /opt/poetry/bin/poetry && \
    poetry config virtualenvs.create false

WORKDIR /app

# Copy using poetry.lock* in case it doesn't exist yet
COPY ./app/poetry.lock* ./app/pyproject.toml /app/
RUN poetry install --no-root --no-dev

COPY ./app/app/example.py /app/

# Copy hello-cron file to the cron.d directory
COPY ./app/cronjob /etc/cron.d/hello-cron

# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/hello-cron
RUN chmod a+x /app/example.py

# Apply cron job
RUN crontab /etc/cron.d/hello-cron

# Create the log file to be able to run tail
RUN touch /var/log/cron.log

# Run the command on container startup
CMD cron && tail -f /var/log/cron.log

The cronjob file that is being used to create the crontab:

# Example crontab
* * * * * /app/example.py 2>&1
* * * * * echo $(date) >> /app/example.log 2>&1
# empty line

And the simple Python script (example.py):

#!/usr/local/bin/python
from loguru import logger
from pathlib import Path

f = Path("/app/info.log")
f = str(f) if f.exists() else "./info.log"

logger.add(
    f,
    format="{time:YYYY-MM-DD at HH:mm:ss} {level} {message}",
    level="INFO",
    rotation="10 MB",
)
logger.info("working")

I have tried several different options to get the python script to run every minute, such as removing the shebang and explicitly specifying the python path (/usr/local/bin/python) in the crontab, but have had no luck. Interestingly, the second line of the crontab that writes the time to /app/example.log works as expected. Additionally, manually running the script from the container's bash shell also works and creates the /app/info.log file.

Are there any obvious issues with the Dockerfile/crontab setup?

ColinB
  • 528
  • 3
  • 12
  • 1
    Did you try using `poetry run`? `/opt/poetry/bin/poetry run python /app/example.py` – MatsLindh Apr 13 '23 at 18:23
  • 1
    If you ssh in to the container and run `crontab -l` to see the active cron entries, what does it show? – John Gordon Apr 13 '23 at 18:23
  • @MatsLindh - That doesn't seem to make a difference. Running the command from the container's bash shell works as expected. – ColinB Apr 13 '23 at 18:44
  • @JohnGordon - `crontab -l` lists exactly what is in my `cronjob` file above. – ColinB Apr 13 '23 at 18:45
  • `CMD cron && tail -f /var/log/cron.log` This isn't working to read the logs. Looking at cron under `strace`, it doesn't log to that file - only to /dev/log. – Nick ODell Apr 13 '23 at 18:56
  • You can listen to /dev/log using `apt-get install netcat-openbsd` then `nc -lkU /dev/log` – Nick ODell Apr 13 '23 at 18:57
  • If you ssh in to the container and type `/app/example.py` what happens? – John Gordon Apr 13 '23 at 19:01
  • Well it turns out I was making a dumb mistake... There was an error occurring before the logger was defined. The cronjob was working but I wasn't getting any logging information to see that it actually ran. Thank you for all the suggestions! – ColinB Apr 13 '23 at 19:17

1 Answers1

0

Well, I think the issue was half me making a silly mistake, and half not configuring my cronjob correctly.

I tested changing my cronjob that was working to:

* * * * * echo $(date) $(which python) >> /app/example.log 2>&1

and noticed that python's location wasn't found when run in the cron environment. This lead me to change my python line in the crontab to:

* * * * * cd /app && /usr/local/bin/python ./example.py 2>&1

which ended up fixing things in the example snipped I posted.

When applying the update to my actual code, the cronjob was still not running. I eventually found that the cronjob was working, but an error was occurring before the logger was defined. Fixing that bug made everything work as expected.

ColinB
  • 528
  • 3
  • 12