5

I have a docker Dockerfile in which I want to clone a Git repo. If the Git repo is not updated I would like Docker to use its cache. If the repo is updated then building the Docker Image should use the Git repo updates.

Based on this stackoverflow question:

How to prevent Dockerfile caching git clone

I have added the following to my Dockerfile:

#invalidate cache if micropython is updated
ADD https://api.github.com/repos/dwjbosman/micropython/git/refs/heads/master /root/micropython_version.json

RUN git clone --recurse-submodules https://github.com/dwjbosman/micropython.git /root/micropython && \
    cd /root/micropython && \
    git submodule update

I use a makefile with targets to build the image. First I pull the previous image from docker hub:

docker pull $(REGISTRY)/$(DOCKER_IMAGE)

and then I use this pulled image as a cache for the new build:

docker build --cache-from $(DOCKER_IMAGE) -t $(DOCKER_IMAGE)

Afterwards the new image is pushed to Docker hub. I am building the images in a CI environment (Gitlab). If I check the logs the ADD step always invalidates the cache (and the checksum changes) even if no updates were done in the Git repo.

I have checked the output of "https://api.github.com/repos/dwjbosman/micropython/git/refs/heads/master". The contents does not seem to change.

Is there some hidden property (time perhaps?) which still changes?

dwjbosman
  • 906
  • 9
  • 33
  • 1
    I don't have issues running this locally with the classic builder (though buildkit has issues that would break the build). Are you sure the layers exist in the builder from the previous build, or is Gitlab using ephemeral build nodes? – BMitch Oct 28 '19 at 01:04
  • I updated the question and added the image build steps – dwjbosman Oct 28 '19 at 08:25
  • Maybe something with the permission of the file, like in this one?: https://devops.stackexchange.com/a/4812 – MrBerta Oct 29 '19 at 16:06
  • How far down in your Dockerfile is the ADD line? Could it be that your layers are not being reused because `--cache-from` doesn't include the the base image? -- https://github.com/moby/moby/issues/20316#issuecomment-358260810 – UrsaDK Apr 29 '20 at 10:04

0 Answers0