I am looking to run a dockerized python script once every minute and I would like to understand how to make this script run every minute and would I do it via cron?
It will be collecting data via an api call and loading to a mysql db in another docker container I have set up, but for now as an example here is what I have running:
Dockerfile:
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "python", "./loading_data.py" ]
loading_data.py:
print('i am loading data to mysql...')
I have my docker python image built:
REPOSITORY TAG IMAGE ID CREATED SIZE
docker_python latest f7222d16bbce 3 minutes ago 938MB
If I run the image it will execute the printing statement upon being built correct? So do I want to:
write into the python script an infinite loop and have my OS start the docker container and restart it if it fails? (feels wrong)
do I want to have a cron job start the container, destroy it and restart it every minute? (feels excessive)
Or is there a simpler more elegant and better way?
--------------------------------------------------Attempt:
I've been trying to setup the python script as a cron job as suggested following https://github.com/cheyer/docker-cron example here is my build:
Dockerfile:
#python install----------
FROM python:3.6
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "python", "./loading_data.py" ]
#cron install----------
FROM ubuntu:latest
# Install cron
RUN apt-get update
RUN apt-get install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
# Add shell script and grant execution rights
ADD script.sh /script.sh
RUN chmod +x /script.sh
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD cron && tail -f /var/log/cron.log
script.sh:
python ./loading_data.py >> /var/log/cron.log 2>&1
crontab:
* * * * * root /script.sh
# An empty line is required at the end of this file for a valid cron file.
Built and ran the container and the cron job runs no problem every minute, but python can't be found within the container. How would I go about repairing this issue? I believe I am very close:
sudo docker exec -i -t 3c54ffaf2674db1ab0751abe93ce77956d8b5b594a7c48cc589c4841761d4e71 /bin/bash
root@3c54ffaf2674:/# cat /var/log/cron.log
/script.sh: 1: /script.sh: python: not found
/script.sh: 1: /script.sh: python: not found
/script.sh: 1: /script.sh: python: not found