How to run the dockerfile commands only when the source docker image has an update

Question

I use some images straight from docker hub and for some I run a few additional commands.
From time to time I want to upgrade all my containers to use the latest image available.
For the containers I run some additional commands how can I run the dockerFile commands only when the source image has a new version available?

To clarify, here is an example

• /opt/test/docker/docker-compose.yaml

version: '3.0'

services:
  service1:
    container_name: service1
    image: alpine
  service2:
    container_name: service2
    build:
      context: .
      dockerfile: service2.Dockerfile

• /opt/test/docker/service2.Dockerfile

FROM alpine:latest
RUN apk add --no-cache mysql-client
ENTRYPOINT ["mysql"]

I first start the containers with
docker-compose --project-directory /opt/test/docker up --detach

Pulling service1 (alpine:)...
latest: Pulling from library/alpine
213ec9aee27d: Pull complete
Digest: sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad
Status: Downloaded newer image for alpine:latest
Building service2
Sending build context to Docker daemon  3.072kB
Step 1/3 : FROM alpine:latest
 ---> 9c6f07244728
Step 2/3 : RUN apk add --no-cache mysql-client
 ---> Running in 87e2a5c87b22
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
(1/7) Installing mariadb-common (10.6.9-r0)
(2/7) Installing libgcc (11.2.1_git20220219-r2)
(3/7) Installing ncurses-terminfo-base (6.3_p20220521-r0)
(4/7) Installing ncurses-libs (6.3_p20220521-r0)
(5/7) Installing libstdc++ (11.2.1_git20220219-r2)
(6/7) Installing mariadb-client (10.6.9-r0)
(7/7) Installing mysql-client (10.6.9-r0)
Executing busybox-1.35.0-r17.trigger
OK: 39 MiB in 21 packages
Removing intermediate container 87e2a5c87b22
 ---> 6d5f43603eb0
Step 3/3 : ENTRYPOINT ["mysql"]
 ---> Running in cb27de950226
Removing intermediate container cb27de950226
 ---> 1d8ca02b8bb7
Successfully built 1d8ca02b8bb7
Successfully tagged docker_service2:latest
WARNING: Image for service service2 was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating service1 ... done
Creating service2 ... done

When I want to update them, I stop and remove them with
docker-compose --project-directory /opt/test/docker down --remove-orphans

Removing service2 ... done
Removing service1 ... done

, then I start them with build
docker-compose --project-directory /opt/test/docker up --detach --build

Building service2
Sending build context to Docker daemon  3.072kB
Step 1/3 : FROM alpine:latest
 ---> 9c6f07244728
Step 2/3 : RUN apk add --no-cache mysql-client
 ---> Using cache
 ---> 6d5f43603eb0
Step 3/3 : ENTRYPOINT ["mysql"]
 ---> Using cache
 ---> 1d8ca02b8bb7
Successfully built 1d8ca02b8bb7
Successfully tagged docker_service2:latest
Creating service1 ... done
Creating service2 ... done

But the dockerFile commands for service2 are run again even though the Alpine image version has not changed.
How can avoid this build and keep the built image I already had?
Does docker-compose has a built-in mechanism for this?

Answer 1

When the build output says

 ---> Using cache

Docker isn't actually re-running the Dockerfile commands. The build cache has some fairly straightforward rules. In this context, if you're starting from an image or layer that already exists, and you RUN an identical command, Docker re-uses the resulting image/layer without actually rebuilding it. If you look carefully at the hex IDs in the output you should notice they stay the same across rebuilds, and if you look at docker images you won't actually see new images being built.

If you rebuild an image and it comes entirely from cache, it is very fast. In general I'd suggest that the easiest way to rebuild an input only if its base or source files have change is to just unconditionally run docker-compose build, since you'll get the same image if nothing has changed.

... only when the source image has a new version available?

There's one more subtlety to this. When your Dockerfile says FROM alpine:latest, if you already have that image locally, Docker just uses it without checking Docker Hub. The various "build" commands generally have a --pull option that forces Docker to check if the base image has updated. This will download a listing of the layers in the image (the image manifest) but then, if nothing has changed, Docker will skip the actual download.

So you should run something like

docker-compose build --pull
docker-compose up -d

and the output will look very much like your last output, but it will still run fairly quickly; technically you are re-running the "build the image" step but if it comes entirely from cache then you're not getting a new image at all.