How to get docker-compose to always re-create containers from fresh images?

Question

My docker images are built on a Jenkins CI server and are pushed to our private Docker Registry. My goal is to provision environments with docker-compose which always start the originally built state of the images.

I am currently using docker-compose 1.3.2 as well as 1.4.0 on different machines but we also used older versions previously.

I always used the docker-compose pull && docker-compose up -d commands to fetch the fresh images from the registry and start them up. I believe my preferred behaviour was working as expected up to a certain point in time, but since then docker-compose up started to re-run previously stopped containers instead of starting the originally built images every time.

Is there a way to get rid of this behaviour? Could that way be one which is wired in the docker-compose.yml configuration file to not depend "not forgetting" something on the command line upon every invocation?

ps. Besides finding a way to achieve my goal, I would also love to know a bit more about the background of this behaviour. I think the basic idea of Docker is to build an immutable infrastructure. The current behaviour of docker-compose just seem to plain clash with this approach.. or do I miss some points here?

`docker system prune -a` – Ronnie Royston May 13 '23 at 22:30 — Ronnie Royston, May 13 '23 at 22:30

score 423 · Accepted Answer · edited Aug 20 '16 at 05:15

423

docker-compose up --force-recreate is one option, but if you're using it for CI, I would start the build with docker-compose rm -f to stop and remove the containers and volumes (then follow it with pull and up).

This is what I use:

docker-compose rm -f
docker-compose pull
docker-compose up --build -d
# Run some tests
./tests
docker-compose stop -t 1

The reason containers are recreated is to preserve any data volumes that might be used (and it also happens to make up a lot faster).

If you're doing CI you don't want that, so just removing everything should get you want you want.

Update: use up --build which was added in docker-compose 1.7

edited Aug 20 '16 at 05:15

Stefan Hanke

3,458
2
30
34

answered Sep 16 '15 at 20:40

dnephin

25,944
9
55
45

1

Yeah, actually this is what I do in CI as well. Not sure why I didn't mention that... – Adrian Mouat Sep 16 '15 at 21:13
@dnephin `docker-compose run -d` doesn't exists ? You want to say `docker-compose up -d` no ? – Guillaume Vincent Apr 07 '16 at 06:14
2

if you run ```docker-compose pull``` before ```docker-compose rm -f``` you can save even more time – stephanlindauer Oct 14 '16 at 14:43
`rm` removes containers, and `pull` pulls images. I don't see how that would save more time. Could you elaborate ? – dnephin Oct 14 '16 at 18:33
3

What is the the -d flag at the end do? – David J. Davis Jan 12 '17 at 15:44
4

"-d Detached mode: Run containers in the background," – dnephin Jan 13 '17 at 16:47
1

It may also be worth mentioning that is you are using a Dockerfile then you may need to build with the `--pull` option as the `docker-compose pull` command will not do this. So there may be another step running `docker-compose build --no-cache --pull` and then simply `docker-compose up -d` – Daniel West May 21 '18 at 15:39
4

This did not work in Mac OS. – Danny Bullis Nov 05 '21 at 06:55

score 395 · Answer 2 · edited May 02 '22 at 05:46

395

The only solution that worked for me was the --no-cache flag:

docker-compose build --no-cache

This will automatically pull a fresh image from the repo. It also won't use the cached version that is prebuilt with any parameters you've been using before.

edited May 02 '22 at 05:46

Jens Habegger

5,266
41
57

answered Mar 23 '17 at 07:57

davidbonachera

4,416
1
21
29

1

In addition, under Windows 10 it can help to set the DNS server in the settings from Automatic to Fixed or from Fixed to Automatic. – qräbnö Apr 15 '18 at 15:42
3

Worked for me on OS X building with docker-comopse version 2. – RoboBear Aug 05 '18 at 23:19
2

Worked on OS X docker. – ABCD Sep 28 '19 at 15:13
1

This worked for me in Mac OS as well. The accepted answer did not. – Danny Bullis Nov 05 '21 at 06:55
2

This does not work in docker-compose 1.29.2. – Rob Feb 12 '22 at 15:28

score 134 · Answer 3 · edited Jan 11 '23 at 21:50

134

By current official documentation there is a shortcut that stops and removes containers, networks, volumes, and images created by up, if they are already stopped or partially removed and so on, then it will do the trick too:

docker-compose down

Then if you have new changes on your images or Dockerfiles use:

docker-compose build --no-cache

Finally:docker-compose up

In one command: docker-compose down && docker-compose build --no-cache && docker-compose up

edited Jan 11 '23 at 21:50

danronmoon

3,814
5
34
56

answered May 24 '17 at 08:07

Victor Timoftii

2,717
1
16
15

4

`docker-compose build --no-cache` is needed only if there are changes on Dockerfiles. – Victor Timoftii Apr 23 '19 at 13:53
Indeed, Victor. Thanks! I thought that it was also necessary after updating a module/application that is executed when the container starts up. For these cases, before running `docker-compose up`, is necessary to rebuild the services with `docker-compose build`. – ivanleoncz Apr 23 '19 at 14:09

score 64 · Answer 4 · edited Dec 06 '22 at 07:08

64

docker-compose up --build  # still use image cache

OR

docker-compose build --no-cache  # never use cache

edited Dec 06 '22 at 07:08

Nam G VU

33,193
69
233
372

answered May 21 '20 at 08:53

flgn

789
6
12

24

When possible, please make an effort to provide additional explanation instead of just code. Such answers tend to be more useful as they help members of the community and especially new developers better understand the reasoning of the solution, and can help prevent the need to address follow-up questions. – Rajan May 21 '20 at 09:55
6

Actually this answer was the most clean and clear to me. – Gigino Mar 17 '21 at 08:53
3

My understanding is that the --build command rebuilds, but uses cached parts, and thus not fully "recreates" the containers. Only the second does what I want. – Olindholm Jun 11 '21 at 08:58
Doesnt work on docker on centos 7, had to rm -rf /var/lib/docker – Nick M Jul 09 '21 at 21:17
7

Two commands are completely different! `docker-compose up --build` uses cache images if it is exist. `docker-compose build --no-cache` never uses cache. – Maxim Mandrik Nov 24 '21 at 10:41
`up --build` still USE CACHE - sorry this is NOT an answer for me – Nam G VU Dec 06 '22 at 07:07
`build --no-cache` is the one – Nam G VU Dec 06 '22 at 07:07

Adrian Mouat · Answer 5 · 2015-09-16T16:00:18.810

36

You can pass --force-recreate to docker compose up, which should use fresh containers.

I think the reasoning behind reusing containers is to preserve any changes during development. Note that Compose does something similar with volumes, which will also persist between container recreation (a recreated container will attach to its predecessor's volumes). This can be helpful, for example, if you have a Redis container used as a cache and you don't want to lose the cache each time you make a small change. At other times it's just confusing.

I don't believe there is any way you can force this from the Compose file.

Arguably it does clash with immutable infrastructure principles. The counter-argument is probably that you don't use Compose in production (yet). Also, I'm not sure I agree that immutable infra is the basic idea of Docker, although it's certainly a good use case/selling point.

edited Sep 16 '15 at 16:00

answered Sep 16 '15 at 15:54

Adrian Mouat

44,585
16
110
102

Thanks for the answer. I think it would be really useful to force it at the configuration level, eg. to enforce it for a database container and disable recreation by default for the app containers.. – Kristof Jozsa Sep 17 '15 at 08:06
17

`--force-recreate` doesn't work for me ... Image is not pulled even though a newer version is out there... – lisak May 24 '16 at 12:09
4

@lisak I never said it pulled new images. It doesn't. It just starts new containers using whatever image is locally available. You'll need to run docker pull manually. – Adrian Mouat Aug 20 '16 at 13:43

score 7 · Answer 6 · answered Jun 30 '22 at 01:21

7

docker-compose up --build --force-recreate

answered Jun 30 '22 at 01:21

Pawel

16,093
5
70
73

With what version of docker compose this command works? – lubrum Aug 29 '23 at 15:18

score 6 · Answer 7 · answered Jul 26 '20 at 18:18

6

I claimed 3.5gb space in ubuntu AWS through this.

clean docker

docker stop $(docker ps -qa) && docker system prune -af --volumes

build again

docker build .

docker-compose build

docker-compose up

answered Jul 26 '20 at 18:18

OurangZeb Khan

1,114
18
20

score 5 · Answer 8 · answered Aug 31 '22 at 06:26

together with --force-recreate, you might want to consider using this flag too:

 -V, --renew-anon-volumes   Recreate anonymous volumes instead of retrieving
                            data from the previous containers.

I'm not sure from which version this flag is available, so check your docker-compose up --help if you have it or not

score 3 · Answer 9 · answered Oct 05 '21 at 19:42

3

Also if the compose has several services and we only want to force build one of those:

docker-compose build --no-cache <service>

answered Oct 05 '21 at 19:42

obotezat

1,041
16
20

This does not work in docker-compose 1.29.2. – Rob Feb 12 '22 at 15:28

score 1 · Answer 10 · answered May 20 '23 at 22:41

For me in Gitlab-ci when I deployed the project to the server, it has needed to run docker-compose down, then docker system prune for removing caches and again run docker-compose up -d.

But I've just added docker-compose up --force-recreate and it did all of the above commands together. iI worked for me!

score -20 · Answer 11 · answered Jan 31 '16 at 13:25

-20

$docker-compose build

If there is something new it will be rebuilt.

answered Jan 31 '16 at 13:25

Mathias Asberg

3,562
6
30
46

How to get docker-compose to always re-create containers from fresh images?

11 Answers11

Linked