'docker-compose' creating multiple instances for the same image

Question

I need to start multiple containers for the same image. If I create my compose file as shown below, it works fine.

version: '2'

services:
  app01:
    image: app
  app02:
    image: app
  app03:
    image: app
  app04:
    image: app
  app05:
    image: app

Is there an easy way for me to mention the number of instances for the compose instead of copy and pasting multiple times?

Bernard · Accepted Answer · 2023-07-03T08:34:33.580

159

Updated answer (Jul 2023)

The old "scale" feature is now called replicas and it is part of the current Docker Compose specs, 2.19.1 as of writing.

Note that replicas is ignored if you name your container using container_name: myname. You must let the Docker generate the names.

services:
  myapp:
    image: awesome/webapp
    deploy:
      mode: replicated
      replicas: 6

Updated answer (Oct 2017)

As others mentioned, the Docker API has changed. I'm updating my answer since it's the one most people will probably look at.

docker-compose up -d --scale app=5

Unfortunately, we cannot specify this in a docker-compose.yml file currently as of version 3.5 (1.18.0 in re-numbered version).

Details:
They did introduce the scale option for version 2.2 and 2.3 of docker-compose, but they removed it for version 3.0. Also, to use version 2.2 or 2.3 you would need to download an older version of the docker-compose tool. The current version does not support 2.2 or 2.3 (it does support 2.0 or 2.1 however). There is also a new deploy section with replicas: 5 but it's only for swarm mode.

Old Answer

docker-compose scale app=5

See docker compose up.

Then you only need this docker-compose file:

version: '2'

services:
  app:
    image: app

edited Jul 03 '23 at 08:34

answered Sep 23 '16 at 14:29

Bernard

16,149
12
63
66

11

Is there a way to specify this in the docker-compose file, so that when you run `docker-compose up`, it will run the preconfigured number of instances? – augustomen Oct 13 '17 at 14:09
1

How do you set the port range for the containers? – Muhammad Abrar Mar 11 '18 at 14:13
1

@MuhammadAbrar You specify the ports the same as with any service. Either use `expose:` for exposing ports to other services on the same network or `ports:` to expose ports to the host (publicly accessible). https://docs.docker.com/compose/compose-file/#short-syntax-1 – Bernard Mar 13 '18 at 07:53
1

Notice that omitting `-d` (daemon mode) means *all* containers started using the same `docker-compose` file will stop once Ctrl+C is used to terminate the last scaled container. Very dangerous. I'd suggest adding `-d` to `docker-compose up` in the answer above, to avoid copy/paste disasters. – Marco Chiappetta Mar 28 '19 at 13:22
1

@MarcoChiappetta good point. I usually do `up -d`. Typo this time. – Bernard Mar 29 '19 at 01:26
@Bernard How to ensure that each instance has a different port exposed to the host? – xkcd Jan 12 '21 at 15:55
1

@xkcd You can use "expose: 4000" instead of "expose: 4000:4000". This will give each instance its own IP address on the internal network. The Docker daemon has a DNS resolver which will round robin the requests to port 4000 on the host to one of the instances. In practice though, you would put a proper load balancer in front (nginx, haproxy, ...) and not expose your instance to the outside world. – Bernard Jan 14 '21 at 03:26
1

Is this answer still valid? – alper Jun 30 '23 at 09:50
1

thanks for the heads up @alper. I'm not following Docker as closely as I used to so my answer was seriously outdated and Docker Compose has a horrible track record for backward compatibility. – Bernard Jul 02 '23 at 04:13

score 73 · Answer 2 · edited Mar 26 '23 at 16:18

73

You can do it with replica as mentioned in Compose specification:

version: '3'
services:
  worker:
    image: dockersamples/examplevotingapp_worker
    networks:
      - frontend
      - backend
    deploy:
      mode: replicated
      replicas: 6

One can use docker-compose --compatibility up to make Docker accept a deploy section without using swarm.

edited Mar 26 '23 at 16:18

Gio

2,242
1
16
13

answered Jul 01 '17 at 17:08

Esra

1,062
8
5

36

Replicas is only for deploying in a swarm. – Ken J Jul 17 '17 at 01:28
@KenJ does that make a functional difference? – Dave Lawrence Oct 31 '17 at 12:49
3

Yes, if the option is ignored by ordinary `docker-compose up` commands. – giorgiosironi Jan 19 '18 at 15:13
I don't see in the linked docs that this is ignored by docker-compose up? All I see is that creating a replica is the default. – Marviel Jan 23 '18 at 16:42
3

The `deploy` intro/header docs state that the entire section is ignored by `docker-compose up` (https://docs.docker.com/compose/compose-file/#deploy) – hcarty Jan 26 '18 at 00:33
So using a standalone docker-compose without swarn, I have no chance to start multiple instance without declaring multiple containers in the compose-file? – Lion Mar 01 '18 at 19:19
42

One can use `docker-compose --compatibility up` to make docker accept `deploy` section without using swarm. – Nikola Petkanski Oct 08 '18 at 19:12
3

@NikolaPetkanski your comment should be marked as the correct answer. Thanks! – Rinor Feb 10 '19 at 21:41
@NikolaPetkanski : that is NOT the correct answer. I get "ERROR: for database Cannot create container for service database: Conflict. The container name "/mysql" is already in use by container " – Shōgun8 Nov 01 '19 at 21:37
@Octavian using `docker-compose --compatibility up`, no providing the container name will work – Daisy QL Dec 17 '19 at 19:09
The `replica` documentation *may* have moved to [another page](https://docs.docker.com/compose/compose-file/deploy/#replicas). – Peter Mortensen Nov 26 '22 at 13:31
Can we assign different port numbers for each created replica? – alper Jun 30 '23 at 16:39

score 44 · Answer 3 · edited Nov 29 '22 at 13:45

44

The scale command is now deprecated, and you should use up instead.

docker-compose up --scale app=2

More details are on docker compose up.

edited Nov 29 '22 at 13:45

Peter Mortensen

30,738
21
105
131

answered Jun 29 '17 at 16:20

fssilva

995
11
7

This is actually problematic and it's not covered in the doc. If we have services A, B, C and we do docker-compose up --scale B=2, it's going to start all A, B, C, so it's equivalent to docker-compose up --scale A=1 --scale B=2 --scale C=1. I'm pretty sure this is not what the user wants in many cases. – Shiva Wu Nov 07 '18 at 21:10
1

@fssilva, This command worked in my case. However I have to modify my compose file. (1) remove container name from compose file to resolve "name is already in use by container' error. (2) remove explicit port bindings to resolve "failed: port is already allocated" error. – harshit2811 Jul 13 '20 at 07:38
1

@ShivaWu in that case specify the desired service at the end e.g.`docker-compose up --scale appB=2 appB` to only start service B. I normally do `docker-compose up` then if I want additional instances of a particular service I follow up with `docker-compose up --scale appB=5 appB` – Gostega Jan 14 '21 at 00:14

score 10 · Answer 4 · edited Dec 08 '22 at 21:49

10

You can do this:

version: "3.4"

services:
  service1: &service_1
    image: app

  service2:
    <<: *service_1

  service3:
    <<: *service_1

For more information on <<, see What is the << (double left arrow) syntax in YAML called, and where's it specified?.

edited Dec 08 '22 at 21:49

Peter Mortensen

30,738
21
105
131

answered Feb 18 '22 at 18:16

Lenny4

1,215
1
14
33

1

This tricky basically defined multiple services. Only applicable for a worker container, if your container provide service to others, supposed to `expose` port, the other containers will not able to connect via service discovery. – Dai Apr 13 '22 at 06:46
If you are using multiple docker-compose files (like when calling docker-compose with the -f flag or setting the COMPOSE_FILE env variable), the anchor must be defined in the same file you are referencing it, can't have them splitted in different files. – Alvaro Flaño Larrondo Jun 06 '23 at 17:38

score 2 · Answer 5 · answered Dec 30 '22 at 16:16

2

Works for me well:

version: "3.9"

services:
  web:
    image: redis:6.2-alpine
    ...
    deploy:
      mode: replicated
      replicas: 3

and run the command then:

docker-compose --compatibility up

answered Dec 30 '22 at 16:16

Vova

3,117
2
15
23

'docker-compose' creating multiple instances for the same image

5 Answers5

Updated answer (Jul 2023)

Updated answer (Oct 2017)

Old Answer

Linked