0

How could one use Docker Compose to synchronize container execution?

The problem I'm trying to solve is similar to Docker Compose wait for container X before starting Y. I use Docker Compose to launch several containers, all running on the same host, three of which are PostgreSQL, Liquibase, and a Web application servlet running in Tomcat. The PostgreSQL and Web application containers are both long running while the Liquibase container is ephemeral. The containers must not only start in order, but each container must also wait for the preceding container to be available or complete. In particular, the PostgreSQL server must be ready to process SQL commands before the Liquibase container runs, and the Liquibase schema migration task must complete before the Web application starts to ensure that the database schema is in a valid state.

I understand that I can achieve this synchronization using two wrapper "wait-for" scripts that poll for certain conditions (and this may be the only available option), the first of which would poll the availability of the PostgreSQL server to process commands while the second, which would run just prior to the Web application, could poll for the presence of a particular database object. However, like process synchronization, I think container synchronization is a common problem that can be addressed with more general inter-process communication and synchronization primitives like semaphores. Docker Compose would likely benefit the most from such synchronization mechanisms, but Docker containers might find them useful, too, for example, to establish multiple synchronization points within a container.

Community
  • 1
  • 1
Derek Mahar
  • 27,608
  • 43
  • 124
  • 174
  • There's something strange about this question. A container is just a process. The fact that the process lifecycle is managed by docker doesn't matter. Are you asking what's a simple IPC mechanism for distributed software? – Alfred Rossi May 05 '16 at 23:35
  • I don't agree with your premise that "a container is just a process". If a container were just a process, we wouldn't need containers. The intent of my question is to find a simple, standard Docker (or Docker Compose) specific solution to problems similar to http://stackoverflow.com/questions/31746182/docker-compose-wait-for-container-x-before-starting-y and https://github.com/docker/compose/issues/374. I was hoping that Docker might already have an IPC mechanism that I didn't know about. – Derek Mahar May 06 '16 at 01:35
  • 1
    Some container orchestration systems like docker-compose allow you to specify a depends-on relationship so that you get a meaningful startup order. However this is somewhat hacky. Setups like this tend not to be fault tolerant in the sense that a process becomes stuck if some dependency fails. Well designed software will usually handle it's own service discovery and fault recovery via a service like zookeeper, etcd, or consul. What specifically are you trying to accomplish? The devil is in the details here. – Alfred Rossi May 06 '16 at 02:05
  • I've rephrased my question and included the particular problem that I'm trying to solve. – Derek Mahar May 06 '16 at 15:48
  • An alternative approach to using a synchonization mechanism would be to combine or bind the PostgreSQL and Liquibase containers (or the Liquibase and Web application containers) so that each runs in sequence, but neither Docker Compose nor Docker provides such a facility. Container composition, in addition to single inheritance, would be useful in many ways. – Derek Mahar May 06 '16 at 17:55
  • I just voted to close my question because I now realize that it really is too broad and open-ended. – Derek Mahar May 13 '16 at 16:51

3 Answers3

0

Until Docker Compose or Docker supports container synchronization primitives (similar to process synchronization primitives, but accessible from the shell), Dependencies for docker-compose with inotify is one of the better solutions that I've found to the Docker Compose container synchronization problem.

Derek Mahar
  • 27,608
  • 43
  • 124
  • 174
0

In addition to consul, etcd, and ZooKeeper, MQTT retained messages are another simple mechanism that Docker containers might use to coordinate activities. Mosquito is a lightweight, open-source implementation of MQTT.

Derek Mahar
  • 27,608
  • 43
  • 124
  • 174
0

I've come to the conclusion that Docker Compose is not the most appropriate tool for container synchronization. Tools like Kubernetes or Marathon facilitate more sophisticated container synchronization. What is the best Docker Linux Container orchestration tool? compares available container synchronization tools.

Derek Mahar
  • 27,608
  • 43
  • 124
  • 174