How to run different python scripts in parallel containers from same image

Question

I am trying to do something very simple (I think), I want do build a docker image and launch two different scripts from the same image in parallel running containers.

Something as simple as

Container 1 -> print("Hello")

Container 2 -> print('World")

I did some research but some techniques seem a little over engineered and others do something like

CMD ["python", "script.py"] && ["python", "script2.py"]

which, isn't what I'm looking for.

I would love to see something like

$ docker ps
CONTAINER ID        IMAGE            CREATED          STATUS               NAMES
a7e789711e62        67759a80360c   12 hours ago     Up 2 minutes         MyContainer1
87ae9c5c3f84        67759a80360c   12 hours ago     Up About a minute    MyContainer2

But running two different scripts.

I'm still fairly new to all of this, so if this is a foolish question, I apologize in advance and thank you all for working with me.

It's an idea, not an instruction, hence not posting as an answer :) One option is to use environment variables - since you want to containers to run the same image you'll need two `docker run $image_name` executions. Between the two you can set env vars differently, e.g. `docker run $image -e RUN_SCRIPT=script1.py` and `docker run $image RUN_SCRIPT=script2.py`. Than in dockerfile you'll need to make it recognize the env var and run the corresponding script. Finally, you could "automate" launching these two containers together using docker-compose - but that's totally optional. — J0HN, Jul 04 '19 at 04:19

constt · Accepted Answer · 2019-07-04T05:00:53.973

You can do this easily using Docker Compose. Here is a simple docker-compose.yml file just to show the idea:

version: '3'

services:
  app1:
    image: alpine
    command: >
      /bin/sh -c 'while true; do echo "pre-processing"; sleep 1; done'

  app2:
    image: alpine
    command: >
      /bin/sh -c 'while true; do echo "post-processing"; sleep 1; done'

As you see both services use the same image, alpine in this example, but differ in commands they run. Execute docker-compose up and see the output:

app1_1  | pre-processing
app2_1  | post-processing
app2_1  | post-processing
app2_1  | post-processing
app1_1  | pre-processing
app2_1  | post-processing
app1_1  | pre-processing
app2_1  | post-processing
...

In your case, you just change the image to your image, say myimage:v1, and change the service commands so that the first service definition will have command: python script1.py line and the second one command: python script2.py.

This helped a-lot, this is exactly what I needed. – Chris Jul 04 '19 at 14:37 — Chris, Jul 04 '19 at 14:37

score 2 · Answer 2 · answered Jul 04 '19 at 04:21

If you are set on using the same image then I would suggest you set the ENTRYPOINT to python and then use docker run command to start the containers by providing the scrips as the CMD like so:

Dockerfile:

FROM python

...

ENTRYPOINT ["python"]

And the use the docker run command like so

docker run -d my_image script.py && docker run -d my_image script2.py

Which would start two containers each running separate scripts

BUT - I like to keep the images clean in terms of not having any additional scripts or packages that are not necessary for my service to work, so in this case I would simply create two separate images each having one of the scripts and then run them similarly.

Example:

FROM python

COPY script.py script.py

ENTRYPOINT ["python"]

CMD ["script.py"]

And the second image:

FROM python

COPY script2.py script2.py

ENTRYPOINT ["python"]

CMD ["script2.py"]

And then just build them as separate images and run the the same way as before

score 2 · Answer 3 · answered Jul 04 '19 at 09:16

Any command you put at the end of the docker run command (or the Docker Compose command: field) replaces the CMD in the Dockerfile. I would suggest still putting in some useful default CMD, but you can always just

docker run --name hello myimage python script.py
docker run --name world myimage python script2.py

score 1 · Answer 4 · answered Jul 04 '19 at 04:39

Try these steps, it should work.

Create a Dockerfile with contents:

FROM python:3.7-alpine
COPY script1.py /script1.py
COPY script2.py /script2.py
CMD ["/bin/sh"]

In script1.py

print("Hello")

In script2.py

print("World")

Build docker image docker build -t myimage:v1 .
Run the containers

$ docker run -it --rm --entrypoint python myimage:v1 /script1.py
Hello
$
$ docker run -it --rm --entrypoint python myimage:v1 /script2.py
World
$

NOTE: Here we're using same docker image myimage:v1 and just changing the entrypoint in every docker run command.

More info here.

Hope this helps.

How to run different python scripts in parallel containers from same image

4 Answers4