0

... so that the two containers both run in the same host, but NOT container inside container?

Long version

I have made two python programs called scheduler and worker. The scheduler periodically asks a server if there is any task to do. If there is some coming-soon task the scheduler will download input data and start the worker with the downloaded data. I am now thinking of containerizing the two programs. What I need here include:

  • Make Dockerfiles for the two containers.
  • Two containers should share a common volume (belonging to the host, of course) where the input data is downloaded / used.
  • Scheduler container should be able to start worker container so that the worker container will run in the host environment but NOT inside the scheduler container.

Is there an easy way to do so?

duong_dajgja
  • 4,196
  • 1
  • 38
  • 65
  • Possible duplicate of [Is it ok to run docker from inside docker?](https://stackoverflow.com/questions/27879713/is-it-ok-to-run-docker-from-inside-docker) – eLRuLL Dec 29 '18 at 14:46
  • @eLRuLL OP mentioned "NOT container inside container". – dkb Dec 29 '18 at 14:47
  • @eLRuLL I don't this question is duplicate with the one that you provided. Please notice `NOT container inside container`! – duong_dajgja Dec 29 '18 at 14:51
  • 1
    What @eLRuLL posted is not irrelevant because it points you to the right direction. Containers are processes. How are these processes started/stopped and controlled in general? You use `docker ...` commands and the docker cli talks to `docker.sock`. You can read more here: [Can anyone explain docker.sock](https://stackoverflow.com/questions/35110146/can-anyone-explain-docker-sock). In a similar way, you will have to send the appropriate request to that socket to start a container. The relevant part of the API can be found here: https://docs.docker.com/engine/api/v1.39/#operation/ContainerStart – tgogos Dec 29 '18 at 15:18

1 Answers1

0

You can, but you are trusting a container with unrestricted root access over the host.

When you launch the container that wants to launch other containers, you need to bind mount the host's Docker socket, typically with

docker run -v /var/run/docker.sock:/var/run/docker.sock ...

The Dockerfile should install the standard Docker CLI (or whatever other language-specific Docker API), and then when it makes docker calls it will work exactly as though you had run the same Docker command from the host. In particular that means any docker run -v options from the container use the host's filesystem path. It also means the container can launch a sub-container accessing any file on the host, even in system directories like /etc, and can freely examine the details of any other Docker container that's running.

In the architecture you're describing, a safer path would be to combine the "scheduler" and "worker" parts into a single container. It periodically polls the server, and if there is work to do, does it. If you want to do more work at once, you can launch multiple copies of this container. This also lets you readily limit the amount of parallel work you're doing at once, so you don't try to pull down a thousand jobs you can't possibly all run at once on your 4-core system.

David Maze
  • 130,717
  • 29
  • 175
  • 215