Docker-compose: deploying service in multiple hosts

Question

I have a docker-compose file that deploys 8 different docker services in the same host. Is it possible to deploy it in different hosts?, I would like to deploy some service in one hosts and another ones in other host remotelly. Would I need to use docker-swarm? or is an easier way to do?

I have read that it could be done using DOCKER_HOST, but if I configure /etc/default/docker with this variable, all the services would be run on the remote host, and what I need is some services in one remote host, and other services in other remote host.

score 26 · Answer 1 · answered Sep 13 '17 at 08:37

We can do this with docker compose v3 now.

https://docs.docker.com/engine/swarm/#feature-highlights https://docs.docker.com/compose/compose-file/
You have to initialize the swarm cluster using command

$ docker swarm init
You can add more nodes as worker or manager -

https://docs.docker.com/engine/swarm/join-nodes/
Once you have your both nodes added to the cluster, pass your compose v3 i.e deployment file to create a stack. Compose file should just contain predefined images, you can't give a Dockerfile for deployment in Swarm mode.

$ docker stack deploy -c dev-compose-deploy.yml --with-registry-auth PL
View your stack services status -

$ docker stack services PL
Try to use Labels & Placement constraints to put services on different nodes.

Example "dev-compose-deploy.yml" file for your reference -

version: "3"

services:

  nginx:
    image: nexus.example.com/pl/nginx-dev:latest
    extra_hosts:
      - "dev-pldocker-01:10.2.0.42”
      - "int-pldocker-01:10.2.100.62”
      - "prd-plwebassets-01:10.2.0.62”
    ports:
      - "80:8003"
      - "443:443"
    volumes:
      - logs:/app/out/
    networks:
      - pl
    deploy:
      replicas: 3
      labels:
        feature.description: “Frontend”
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: any
      placement:
        constraints: [node.role == worker]
    command: "/usr/sbin/nginx"

  viz:
    image: dockersamples/visualizer
    ports:
      - "8085:8080"
    networks:
      - pl
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    deploy:
      replicas: 1
      labels:
        feature.description: "Visualizer"
      restart_policy:
        condition: any
      placement:
        constraints: [node.role == manager]

networks:
pl:

volumes:
logs:

BMitch · Accepted Answer · 2019-09-13T16:03:14.877

With docker swarm mode, you can deploy a version 3 compose yml file using:

docker stack deploy -c docker-compose.yml $your_stack_name

The v3 syntax removes a few features that do not apply to swarm mode, like links and dependencies. You should also note that volumes are stored local to a node by default. Otherwise the v3 syntax is very similar to the v2 syntax you may already be using. See ether following for more details:

https://docs.docker.com/compose/compose-file/

https://docs.docker.com/engine/swarm/

[ Original answer before v3 of the docker-compose.yml ]

For a single docker-compose.yml to deploy to multiple hosts, you need to use the standalone swarm (not the newer swarm mode, yet, this is rapidly changing). Spin up a swarm manager that has each host defined as members of its swarm, and then you can use constraints inside your docker-compose.yml to define which services run on which hosts.

You can also split up your docker-compose.yml into several files, one for each host, and then run multiple docker-compose up commands, with a different DOCKER_HOST value defined for each.

In both cases, you'll need to configure your docker installs to listen on the network, which should be done by configuring TLS on those sockets. This documentation describes what you need to do for that.

But this means you still need swarm right? or am I reading it wrong? I am also in the same situation where I would want to deploy 2 services to 2 different hosts. Not sure if only docker-compose can achieve this, or I have to use it with docker swarm. Can you clarify? — Finlay Weber, Nov 25 '19 at 12:23
@FinlayWeber The current best practices with docker is to use Swarm Mode (not to be confused with classic Swarm). It takes the same `docker-compose.yml` input that is also used by `docker-compose` for single node tasks. — BMitch, Nov 25 '19 at 14:27
Thanks for the clarification with the "classic Swarm". I was actually getting confused with that. Thanks! — Finlay Weber, Nov 25 '19 at 17:48

score 7 · Answer 3 · answered May 12 '17 at 14:01

7

You can use docker compose version 3 which provides ability to do multi host deployment without using multiple compose files. All you need is to define labels for each node in the cluster and use the label name for placement constraint.

answered May 12 '17 at 14:01

meswapnilk

73
1
5

But is it need to create a Swarm cluster? Or how can I define the different hosts labels to known between thems? – Asier Gomez May 15 '17 at 06:24
To use the placement feature you will need to use Swarm. – meswapnilk May 16 '17 at 04:14

score 3 · Answer 4 · answered Aug 05 '20 at 04:55

You may also want to consider the Overnode tool - it is container orchestration tool on top of automated multi-host docker-compose. It is the easiest transition from the single-host docker-compose deployment. (Disclaimer: I am an author and it was published recently)

Docker-compose: deploying service in multiple hosts

4 Answers4

Linked