Docker compose does not completely invalidate cache when asked

Question

In an effort to update my container to the newest version of PHP 8.0 (that is 8.0.20 at the time of writing) I have tried running

$ docker compose build --no-cache
[+] Building 148.2s (24/24) FINISHED                                                                                                                                                                                                                                  
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                             0.0s
 => => transferring dockerfile: 1.97kB                                                                                                                                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/php:8.0-apache                                                                                                                                                                                                0.0s
 => [internal] load build context                                                                                                                                                                                                                                0.0s
 => => transferring context: 6.17kB                                                                                                                                                                                                                              0.0s
 => CACHED [base 1/9] FROM docker.io/library/php:8.0-apache                                                                                                                                                                                                      0.0s
 => [base 2/9] RUN a2enmod rewrite 
...

But as seen from the output, only step 2 and up are rebuilt, the base image is still being read from cache, resulting in PHP version 8.0.8.

How can I force a complete rebuild without using old cache?

---

$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu4

$ docker compose version
Docker Compose version v2.4.0

Top of Dockerfile:

FROM php:8.0-apache as base

# Apache rewrite module
RUN a2enmod rewrite

---

EDIT: After more research I find this question is similar to and possibly a duplicate of How to get docker-compose to always re-create containers from fresh images?. The missing part in this specific example is docker pull php:8.0-apache.

I don't understand why though. Why do I have to manually pull the fresh version of the base image?

Pruning (docker system prune, docker builder prune -a) has no effect on this issue, even after taking the containers down.

Answer 1

You can do something like:

docker-compose up --force-recreate

or something like this:

docker-compose down --rmi all --remove-orphans && docker-compose up --force-recreate

This will remove all the images so use at your own discretion. Reference to docker-compose down command here

Answer 2

There's a general Docker rule that, if you already have some image locally, it's just used without checking Docker Hub. As @BMitch indicates in their answer the newer BuildKit engine should validate that you do in fact have the most current version of an image, but it's possible to update this manually.

In your case, you already have a php:8.0-apache image locally (with PHP 8.0.8). So you could manually get the updated base image

docker pull php:8.0-apache
docker-compose build

If the base image has changed, this will invalidate the cache, and so you don't need --no-cache here. This will also work with the "classic" builder (though your output does show that you're using the newer BuildKit engine).

This is a common enough sequence that there's a shorthand for it

docker-compose build --pull

Answer 3

For a base image, CACHED effectively means "verified" where buildkit has queried the registry to check the current base image digest, and seen that the base image digest matches what it's already pulled down. More details are available in this GitHub issue.

There's only 2 reasons I can think of to not use that cache. First is if your local build cache is corrupt. In that case, purge your local cache (docker builder prune). And the other case is a sha256 collision, which if that happens, the registry server itself will probably be broken and fixing the builder is the least concern.

The reason for using --no-cache is to ensure steps that could possibly result in a different output are done again, and that's being done in this example.