7

If I have scripts issueing docker run commands in parallel, the docker engine appears to handle these commands in series. Since runing a minimal container image with "docker run" takes around 100ms to start does this mean issueing commands in parallel to run 1000 containers will take the docker engine 100ms x 1000 = 100 s or nearly 2 minutes? Is there some reason why the docker engine is serial instead of parallel? How do people get around this?

Michael
  • 546
  • 1
  • 7
  • 19
  • It appears that containerd offers some single host startup speed... https://containerd.tools/?mkt_tok=eyJpIjoiTldGbE1EbG1ORGMzTWpFNSIsInQiOiJBR0dhcVJjbENWYnphdDJzRWhLZGcxWXJZS2dndzJNMThaTXdcL1crWU94Y2dXTVhzK05walJlZmNEcHlISHQyWmJWVWw3ZzdKVjZmODArMmljUFM4eGlLbmJkRmJZanJYbG5wVU1PVDg3S3M9In0%3D – Michael Apr 21 '16 at 00:33

1 Answers1

4

How do people get around this?

a/ They don't start 1000 containers at the same time b/ if they do, they might use a cluster management system like docker swarm to manage the all process c/ they do run 1000 containers, in advance in order to take into account the starting time.

Truly parallelize docker run command could be tricky considering some of those command might depend on other containers to be created/started first (like a docker run --volumes-from=xxx)

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • Thanks. The use case I am considering for Docker involves replacing a legacy system of binary applications that involves a fair amount of fork exec operations in c code starting new processes on the fly. I am trying to get a feel for the correct seams to incrementally introduce Docker without bogging down the system. – Michael Jul 17 '15 at 14:28
  • @Mike could those fork be "grouped" amongst several containers, or do you have to have 1 container for one forked process? – VonC Jul 17 '15 at 14:29
  • I think that will be part of the balancing act. A balance of how much work to do in each container instance. There is a very nice logical separation when doing work in containers which is a new benefit and I am trying to reconcile the 1 process per container with how much work do we put in a single processes. – Michael Jul 17 '15 at 14:35
  • What about a solution that uses Docker as a PaaS. You could have DockerNDocker where the internal Docker is Parallel and does not use volumes-from? – Michael Jul 17 '15 at 22:22