10

I have been using docker for a couple of months now, and am working on dockerizing various different server images. One consistent issue is that many servers need to run cron jobs. There is a lot of discussion about that online (including on Stackoverflow), but I don't completely understand the mechanics of it.

Currently, I am using the host's cron and docker exec into each container to run a script. I created a convention about the script's name and location; all my containers have the same script. This avoids having the host's cron depending on the containers.

Basically, once a minute, the host's cron does this:

for each container
   docker exec -it <containername> /cronscript/minute-script

That works, but makes the containers depend on the host.

What I would like to do is create a cron container that kicks off a script within each of the other containers - but I am not aware of an equivalent to "docker exec" that works from one container to the other.

The specific situations I have right now are running a backup in a MySQL container, and running the cron jobs Moodle requires to be run every minute. Eventually, there will be additional things I need to do via cron. Moodle uses command-line PHP scripts.

What is the "proper" dockerized way to kick off a script from one container in another container?

Update: maybe it helps to mention my specific use cases, although there will be more as time goes on.

Currently, cron needs to do the following:

  • Perform a database dump from MySQL. I can do that via mysqldump TCP link from a cron container; the drawback here is that I can't limit the backup user to host 127.0.0.1. I might also be able to somehow finagle the MySQL socket into the cron container via a volume.
  • Perform regular maintenance on a Moodle installation. Moodle includes a php command line script that runs all of the maintenance tasks. This is the biggie for me. I can probably run this script through a volume, but Moodle was not designed with that situation in mind, and I would not rule out race conditions. Also, I do not want my moodle installation in a volume because it makes updating the container much harder (remember that in Docker, volumes are not reinitialized when you update the container with a new image).
  • Future: perform routine maintenance on a number of other of my servers, such as cleaning out email queues, etc.
Kevin Keane
  • 1,506
  • 12
  • 24
  • have you read https://medium.com/@gchudnov/trapping-signals-in-docker-containers-7a57fdda7d86#.uwmlzjix5 – user2915097 Nov 21 '15 at 19:40
  • Fascinating idea, I hadn't thought of doing that. It seems to suffer from the same problem, though; or is there a way for one container to send a signal to another container? – Kevin Keane Nov 21 '15 at 19:48
  • see also http://stackoverflow.com/questions/30545023/how-to-communicate-between-docker-containers-via-hostname and http://blog.sequenceiq.com/blog/2014/08/12/docker-networking/ – user2915097 Nov 21 '15 at 19:56
  • 1
    check the doc `--link=[] Add link to another container` of `docker run` at https://docs.docker.com/engine/reference/commandline/run/ – user2915097 Nov 21 '15 at 19:57
  • Linking doesn't work here, because it only allows network-based communication. The ultimate goal is to kick off a script in the target container, and have the target container run only the one process it is designed for (MySQL, Apache, Nginx). Network communication would require having another process listening for communication from cron. Or am I overlooking something? – Kevin Keane Nov 21 '15 at 20:09
  • If the data you need to operate on is in a volume, you can use either a named volume or `volumes_from`. That lets you run the scheduled task in a different container (same idea as link, but different mechanism). The name of the volume, (or container if you're using volumes from) can be included as part of the crontab entry. – dnephin Nov 21 '15 at 22:16
  • @dnephin - thank you. In some cases, that may work, but in other cases, I may need to interact with processes. I also prefer not to have the cron container have too much information about the other containers - for instance, the cron container shouldn't know what to do with the data in the volumes. That knowledge should all be contained within the specific container. – Kevin Keane Nov 22 '15 at 21:38

1 Answers1

12

My solution is:

  • install crond inside container
  • install Your soft
  • run cron as a daemon
  • run Your soft

Part of my Dockerfile

FROM debian:jessie

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY .crontab /usr/src/app

# Set timezone
RUN echo "Europe/Warsaw" > /etc/timezone \
    && dpkg-reconfigure --frontend noninteractive tzdata

# Cron, mail
RUN set -x \
    && apt-get update \
    && apt-get install -y cron rsyslog mailutils --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

CMD rsyslogd && env > /tmp/crontab && cat .crontab >> /tmp/crontab && crontab /tmp/crontab && cron -f

Description

  1. Set timezone, because cron need this to proper run tasks
  2. Install cron package - package with cron daemon
  3. Install rsyslog package to log cron task output
  4. Install mailutils package if You want to send e-mails from cron tasks
  5. Run rsyslogd
  6. Copy ENV variables to tmp file, because cron run tasks with minimal ENV and You tasks may need access to containers ENV variables
  7. Append Your .crontab file (with Your tasks) to tmp file
  8. Set root crontab from tmp file
  9. Run cron daemon

I use this in my containers and work very well.

one-process-per-container

If You like this paradigm, then make one Dockerfile per cron task. e.g.

  • Dockerfile - main program
  • Dockerfile_cron_task_1 - cron task 1
  • Dockerfile_cron_task_1 - cron task 2

and build all containers:

docker build -f Dockerfile_cron_task_1 ...
Tomasz Jakub Rup
  • 10,502
  • 7
  • 48
  • 49
  • Thank you! I can see how this solution works, but I breaks the one-process-per-container paradigm. I like that this keeps everything contained in the same container, though, and I might actually use this solution; it seems that there is no really good solution for this problem. – Kevin Keane Nov 22 '15 at 21:42
  • I note your addition to the answer (re one process per container), and it does address the separation of concerns, but in the end, it still suffers from the same problem, that the only options for communication from the cron container to the main one is either through shared volumes, or through a link. Your architecture allows cron to have more knowledge of the main container, so it may actually address at least 80% of real-world use cases. – Kevin Keane Nov 30 '15 at 00:19