Can I run multiple programs in a Docker container?

Question

I'm trying to wrap my head around Docker from the point of deploying an application which is intended to run on the users on desktop. My application is simply a flask web application and mongo database. Normally I would install both in a VM and, forward a host port to the guest web app. I'd like to give Docker a try but I'm not sure how I'm meant to use more than one program. The documentations says there can only be only ENTRYPOINT so how can I have Mongo and my flask application. Or do they need to be in separate containers, in which case how do they talk to each other and how does this make distributing the app easy?

Answer 1

There can be only one ENTRYPOINT, but that target is usually a script that launches as many programs that are needed. You can additionally use for example Supervisord or similar to take care of launching multiple services inside single container. This is an example of a docker container running mysql, apache and wordpress within a single container.

Say, You have one database that is used by a single web application. Then it is probably easier to run both in a single container.

If You have a shared database that is used by more than one application, then it would be better to run the database in its own container and the applications each in their own containers.

There are at least two possibilities how the applications can communicate with each other when they are running in different containers:

1. Use exposed IP ports and connect via them.
2. Recent docker versions support linking.

Answer 2

I strongly disagree with some previous solutions that recommended to run both services in the same container. It's clearly stated in the documentation that it's not a recommended:

It is generally recommended that you separate areas of concern by using one service per container. That service may fork into multiple processes (for example, Apache web server starts multiple worker processes). It’s ok to have multiple processes, but to get the most benefit out of Docker, avoid one container being responsible for multiple aspects of your overall application. You can connect multiple containers using user-defined networks and shared volumes.

There are good use cases for supervisord or similar programs but running a web application + database is not part of them.

You should definitely use docker-compose to do that and orchestrate multiple containers with different responsibilities.

More explanation about why it's better to split multiple services in different containers

Due to apparent misunderstanding on this point, I'll make a more complete explanation.

In order to do that I will use a more complex deployment example, as the flask app + mongo db database is a bit too trivial to demonstrate real complexity in modern web application development. Let's imagine we have:

• A Python web server using flask
• A MongoDB server (could be any other database like Postgresl, mariadb, etc..)
• A redis server as shared cache (could be any similar solution like memcached)
• A rabbit MQ server as message queue (could be any other message queue)
• Another Python application that will process messages in the rabbit MQ server
• A caddy instance to serve as reverse proxy and SSL provider (could be Apache, Nginx,...)

So we have 6 type of services. That kind of deployment is nothing new or extraordinary nowadays in web development as most real world, professional projects tend to have at least that complexity (often more).

What Docker will allow us to do here, and the recommended way to use it that is clearly stated in the documentation, is to separate each of these 6 services into separate containers.

Here are the reasons why we will prefer to do so:

Dependencies separation

Each of these services will have its own set of dependencies. At the very least all of them have, directly or indirectly, a dependency on the libc of the system, and even that dependency can be problematic sometimes (some programs are packaged to use more recent versions of the C library). They will also certainly have other dependencies related to the system. That can be low-level C libraries like openssl, libxml, libevent, etc... or more high level tools like Python, nodejs, ruby, etc... And each of those services will only support a range of versions for each of those dependencies.

While you "could" attempt to install each of those services in a single container, that will be more problematic that you could think. You will probably have to rely on the image's base distribution package manager, which means having a specific version imposed for each of those services. You can't choose, as example, the version of postgresql or redis installed by default on a given Ubuntu version. You could also install a specific version of your choice by using third-party repositories but that's usually running into problems on most Linux/Unix systems as you will inevitably run into dependencies incompatibility issues.

Unfortunately being stuck to specific versions of these services is not really a satisfactory solution nowadays. Maybe you would like to use newer version of Postgresql version because there are new features in that version you would like to use in your Python program. Maybe you would need to upgrade the caddy instance due to security issues in the openssl implementation that necessitate a quick upgrade.

Trying to fit specific versions of each of the services you use in a single container will, one day or another, end up being the exact same hell than trying to install those same services together on any Linux machine.

Docker allows to address that problem by allowing each of these services to come with its own dependencies, totally independent from each other. Changing the version of any of those services is usually as simple as changing a single line of configuration in a docker-compose.yml file. That's a huge plus and one of the many reasons to use Docker.

Different scalings

Let's imagine you made your first successful deployment of these 6 services and now your web project is up and running. Now you have a new problem: real persons are using your application. And those real persons use resources. CPU, RAM, disk IOs, network IOs, storage space,... all of that have limitations on the machine you deployed on. You now need to be able to scale your application.

In order to be able to scale an application, you can't just duplicate your whole stack. You can probably scale horizontally for your Python flask application (if it's stateless, as it should be) but scaling a Mongodb instance is completely different and require specific configuration. If you scale your Python application you probably need to adapt your caddy instance to make load balancing. You probably won't scale your redis instance as it's an in-memory cache server. Maybe you only need to scale the background Python application handling messages from the message queue because you figured that's the part of the application that's the most CPU-consuming part. Etc...

Docker allows to do that kind of scaling and configuration adaptations way easier by defining each service in separate containers. In the best cases (not all of them but many) the only adaptations you will need to do is to adapt your Docker configuration without the slightest impact on your images. Scaling a deployment on multiple computers will also be greatly simplified, possibly by using more complex orchestrators like Kubernetes because using docker-compose has limitations but at least you will have solutions to do so.

Different storages

We have a database here, our MongoDB instance. Like all databases it needs special considerations that don't exist in the context of a stateless service. We're speaking about data persistence, backups, archival, data migrations... as these little things that are not so trivial to handle but 100% necessary for even the simplest real-world use case.

To handle these considerations it was common practice in the old days before Docker existed to have guidelines imposing that databases are located on different machines than the ones running application code. These kind of practice still exist today even with the availability of Docker by the way because there are still good reasons to store data completely independently.

But for people that want to use Docker for databases, even if it's only in development, it's still the best thing to do to systematically put them in separate containers. That will allow to manage the lifecycle and storage of the database in a clearly independent way, avoiding any unnecessary and probably unwanted interactions with anything else.

You could also choose to deploy your database with Docker in development, for ease of use, but use something like AWS RDS for your production environment (thus not using Docker). That can also be done easily by just using different configurations and without any side effects on your other services.

Conclusion

When learning to use Docker there are two different aspects that must be mastered:

• Learning how to create new Docker images. This necessitate to learn how to create Dockerfile file and also a lot about sh/bash scripting.
• Learning how to orchestrate existing Docker containers to be able to make good use of Docker for real use cases. Usually we start by learning docker-compose because it's probably the simplest tool for a start but Kubernetes seems to be used more and more for this nowadays (it's also a lot more complex).

These skills are kind of separate, but also heavily intermingled. You need to know how Docker containers work to be able to orchestrate them and for that you need to be able to create Docker images. But to be able to create good Docker images you will also need to understand the end goal of them, which is to be able to orchestrate them easily into multiple containers.

That is why, given the original question, the best advice to give is to not attempt to create a single image for a Python flask application + a MongoDB database. The best advice would be to create a Docker image containing only the Python flask application and create a docker-compose.yml file containing both the Python flask application and the MongoDB database (re-using one of the official MongoDB images.

This will of course necessitate to learn to use docker-compose, learn to configure the official MongoDB image and probably also make some adaptations to the Python flask application to be able to inject some configuration parameters in it through environment variables. But those necessities are far from being a waste of time. These are a full part of the "correct" way to work with Docker to reach its real usefulness.

Answer 3

I had similar requirement of running a LAMP stack, Mongo DB and my own services

Docker is OS based virtualisation, which is why it isolates its container around a running process, hence it requires least one process running in FOREGROUND.

So you provide your own startup script as the entry point, thus your startup script becomes an extended Docker image script, in which you can stack any number of the services as far as AT LEAST ONE FOREGROUND SERVICE IS STARTED, WHICH TOO TOWARDS THE END

So my Docker image file has two line below in the very end:

COPY myStartupScript.sh /usr/local/myscripts/myStartupScript.sh
CMD ["/bin/bash", "/usr/local/myscripts/myStartupScript.sh"]

In my script I run all MySQL, MongoDB, Tomcat etc. In the end I run my Apache as a foreground thread.

source /etc/apache2/envvars
/usr/sbin/apache2 -DFOREGROUND

This enables me to start all my services and keep the container alive with the last service started being in the foreground

Hope it helps

UPDATE: Since I last answered this question, new things have come up like Docker compose, which can help you run each service on its own container, yet bind all of them together as dependencies among those services, try knowing more about docker-compose and use it, it is more elegant way unless your need does not match with it.

Answer 4

Although it's not recommended you can run 2 processes in foreground by using wait. Just make a bash script with the following content. Eg start.sh:

# runs 2 commands simultaneously:

mongod & # your first application
P1=$!
python script.py & # your second application
P2=$!
wait $P1 $P2

In your Dockerfile, start it with

CMD bash start.sh

I would recommend to set up a local Kubernetes cluster if you want to run multiple processes simultaneously. You can 'distribute' the app by providing them a simple Kubernetes manifest.

Answer 5

They can be in separate containers, and indeed, if the application was also intended to run in a larger environment, they probably would be.

A multi-container system would require some more orchestration to be able to bring up all the required dependencies, though in Docker v0.6.5+, there is a new facility to help with that built into Docker itself - Linking. With a multi-machine solution, its still something that has to be arranged from outside the Docker environment however.

With two different containers, the two parts still communicate over TCP/IP, but unless the ports have been locked down specifically (not recommended, as you'd be unable to run more than one copy), you would have to pass the new port that the database has been exposed as to the application, so that it could communicate with Mongo. This is again, something that Linking can help with.

For a simpler, small installation, where all the dependencies are going in the same container, having both the database and Python runtime started by the program that is initially called as the ENTRYPOINT is also possible. This can be as simple as a shell script, or some other process controller - Supervisord is quite popular, and a number of examples exist in the public Dockerfiles.

Answer 6

Docker provides a couple of examples on how to do it. The lightweight option is to:

Put all of your commands in a wrapper script, complete with testing and debugging information. Run the wrapper script as your CMD. This is a very naive example. First, the wrapper script:

#!/bin/bash

# Start the first process
./my_first_process -D
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start my_first_process: $status"
  exit $status
fi

# Start the second process
./my_second_process -D
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start my_second_process: $status"
  exit $status
fi

# Naive check runs checks once a minute to see if either of the processes exited.
# This illustrates part of the heavy lifting you need to do if you want to run
# more than one service in a container. The container will exit with an error
# if it detects that either of the processes has exited.
# Otherwise it will loop forever, waking up every 60 seconds

while /bin/true; do
  ps aux |grep my_first_process |grep -q -v grep
  PROCESS_1_STATUS=$?
  ps aux |grep my_second_process |grep -q -v grep
  PROCESS_2_STATUS=$?
  # If the greps above find anything, they will exit with 0 status
  # If they are not both 0, then something is wrong
  if [ $PROCESS_1_STATUS -ne 0 -o $PROCESS_2_STATUS -ne 0 ]; then
    echo "One of the processes has already exited."
    exit -1
  fi
  sleep 60
done

Next, the Dockerfile:

FROM ubuntu:latest
COPY my_first_process my_first_process
COPY my_second_process my_second_process
COPY my_wrapper_script.sh my_wrapper_script.sh
CMD ./my_wrapper_script.sh

Answer 7

I agree with the other answers that using two containers is preferable, but if you have your heart set on bunding multiple services in a single container you can use something like supervisord.

in Hipache for instance, the included Dockerfile runs supervisord, and the file supervisord.conf specifies for both hipache and redis-server to be run.

Answer 8

If a dedicated script seems like too much overhead, you can spawn separate processes explicitly with sh -c. For example:

CMD sh -c 'mini_httpd -C /my/config -D &' \
 && ./content_computing_loop

Answer 9

In docker, there are two ways you can run a program

1. CMD
2. ENTRYPOINT

If you want to know the difference between them, please refer here

In CMD/ENTRYPOINT, there are two formats to run a command

• SHELL format
• EXEC format

SHELL format:

CMD executable_first arg1; executable_second arg1 arg2
ENTRYPOINT executable_first arg1; executable_second arg1 arg2

This version will create a shell and executes above command. Here you can use any shell syntax such as ";", "&", "|", etc. So you can run any number of commands here. If you have complex set of commands to run, you can create separate shell script and use it.

CMD my_script.sh arg1
ENTRYPOINT my_script.sh arg1

EXEC format:

CMD ["executable", "parameter 1", "parameter 2", …]
ENTRYPOINT ["executable", "parameter 1", "parameter 2", …]

Here you can notice that only first parameter is an executable. From the second parameter, everything become an arguments/parameters for that executable.

To run multiple commands in EXEC format

CMD ["/bin/sh", "-c", "executable_first arg1; executable_second"]
CMD ["/bin/sh", "-c", "executable_first arg1; executable_second"]

In above command, we have used shell command as executable to run the command. This is the only way to run multiple commands in EXEC format.

Following are WRONG

CMD ["executable_first parameter", "executable_second parameter"]
ENTRYPOINT ["executable_first parameter", "executable_second parameter"]

CMD ["executable_first", "parameter", ";", "executable_second",  "parameter"]
ENTRYPOINT ["executable_first", "parameter", ";", "executable_second", "parameter"]

Answer 10

Can I run multiple programs in a Docker container?

Yes. But with significant risks.

---

Below is the same answer as above. But with details and a recommended resolution. If you're interested in those.

Not Recommended

Warning. Using the same container for multiple services is not recommended by the Docker community, though. The Docker documentation reads: "It is generally recommended that you separate areas of concern by using one service per container." Source at:

• https://archive.ph/3Roa6#selection-307.2-307.100

• https://docs.docker.com/config/containers/multi-service_container/

If you choose to ignore the recommendation above, you container risk to be with weaker security, increasingly unstable, and in the future a painful growth.

If you are ok with those risks above, the documentation to use one container for multiple services is at:

• https://archive.ph/3Roa6#selection-335.0-691.1

• https://docs.docker.com/config/containers/multi-service_container/

Recommended

If you need a container(s) with stronger security, and more stability, and in the future, scale bigger, as well as better performance, then the Docker community recommends those two steps:

1. Use one service per Docker container. The end result is that you will have multiple containers.

2. Use this Docker "Networking" feature to connect any of those containers to your liking.