How can I cache Maven dependencies and plugins in a Docker Multi Stage Build Layer?

Question

I want to cache Maven dependencies in a layer of the build stage of my Docker Multi Stage Build.

My Dockerfile looks as follows:

FROM maven:3-jdk-8 as mvnbuild
RUN mkdir -p /opt/workspace
WORKDIR /opt/workspace
COPY pom.xml .
RUN mvn -B -s /usr/share/maven/ref/settings-docker.xml dependency:resolve
COPY . .
RUN mvn -B -s /usr/share/maven/ref/settings-docker.xml package

FROM openjdk:8-jre-alpine
...

```

I based this Dockerfile on the example provided in the Docker Multi Stage Build blog post (also available on Github).

When I run the build, instead of seeing the dependencies downloaded once by dependency:resolve and then re-used by package, I am seeing the dependencies downloaded by both steps.

Has anyone got this working? What am I doing wrong here?

Please look at my solution at https://stackoverflow.com/a/71066133/418599 . — Antonio Petricca, Feb 10 '22 at 14:15

score 10 · Accepted Answer · edited Dec 26 '17 at 14:31

10

I came across the same question. I found out it's due to differences between Maven targets (e.g. dependency:resolve vs dependency:resolve-plugin). Basically, dependency:resolve is for application libraries, dependency:resolve-plugin is for plugin libraries. Hence, libraries are downloaded in both RUN steps.

dependency:resolve tells Maven to resolve all dependencies and displays the version. JAVA 9 NOTE: will display the module name when running with Java 9.

dependency:resolve-plugins tells Maven to resolve plugins and their dependencies.

https://maven.apache.org/plugins/maven-dependency-plugin/index.html

Even with dependency:resolve-plugins, Maven will not download all required libraries as package is a built-in target and requires additional libraries which dependency:resolve-plugin won't know to resolve in the first RUN. I also tried dependency:go-offline without success.

One solution is to run your build targets before and after adding your code to the build image. This will pull all the plugin dependencies into the lower layer allowing them to be re-used.

Applying this solution to your example above is as follows:

FROM maven:3-jdk-8 as mvnbuild
RUN mkdir -p /opt/workspace
WORKDIR /opt/workspace
COPY pom.xml .
RUN mvn -B -s /usr/share/maven/ref/settings-docker.xml dependency:resolve-plugins dependency:resolve clean package
COPY . .
RUN mvn -B -s /usr/share/maven/ref/settings-docker.xml clean package

FROM openjdk:8-jre-alpine

edited Dec 26 '17 at 14:31

Trastle

5,155
6
26
20

answered Dec 25 '17 at 15:43

sayboras

4,897
2
22
40

Any chance you can add the build snippet from your Dockerfile to your answer? I have tried dependency:resolve-plugins, dependency:go-offline and dependency:resolve but I am still seeing dependency (not plugin) downloads on every build. – Trastle Dec 26 '17 at 12:47
1

I can't share the one at work due to company policy. Hence, I am using sample https://github.com/dockersamples/atsea-sample-shop-app for checking only. Please let me know which library (not plugin) got downloaded again in both steps. – sayboras Dec 26 '17 at 13:41
Thanks, I looked again and it was only Plugins. I was thrown off by some of the plugins being cached at jitpack.io – Trastle Dec 26 '17 at 13:59
1

Progress on the caching plugins issue. I am now running: 'dependency:resolve-plugins dependency:resolve clean package' before I COPY my code into the build image then running 'clean package' after copying the code in. The result is that all the required plugins are downloaded by the first RUN and then re-used by the second RUN. The resulting Dockerfile is here: https://github.com/trastle/haverland-smartwave-prometheus-exporter/blob/master/Dockerfile – Trastle Dec 26 '17 at 14:04
Good to know it's working for you. At work, I usually run with `clean package` target. Forgot mentioning this point to you. Sometimes mvn target is confusing. – sayboras Dec 26 '17 at 14:14
3

Thanks for the above snippet. Alas, it did not work with the most recent Spring Boot plugin. Maven was throwing the following error: `[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.0.3.RELEASE:repackage (default) on project core: Execution default of goal org.springframework.boot:spring-boot-maven-plugin:2.0.3.RELEASE:repackage failed: Unable to find main class -> [Help 1]` This is because the Spring Boot plugin tries to repackage the JAR file and make it executable. To fix that error, you have to add `-Dspring-boot.repackage.skip=true`. – Marchev Jul 29 '18 at 19:29

score 8 · Answer 2 · answered Apr 02 '18 at 05:59

I'd like to offer my two cents on this one.

I started with this Dockerfile:

FROM maven:3-jdk-10 AS build

RUN mkdir /src
WORKDIR /src
COPY pom.xml .
RUN mvn -B dependency:resolve-plugins dependency:resolve

COPY . .
RUN mvn package

The goal is to not have any dependencies downloaded during the build step (which is mvn package).

I tried to add the clean package trick that is mentioned in the answer by @Apolozeus but this does not have an effect. In my case, I have surefire being downloaded during test and a mapstruct plugin being downloaded during compilation.

What I did in the end is to explicitly add these two plugins in my pom.xml so that they will be downloaded early on:

    <dependency>
        <groupId>org.apache.maven.surefire</groupId>
        <artifactId>surefire-junit4</artifactId>
        <version>2.21.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.mapstruct</groupId>
        <artifactId>mapstruct-processor</artifactId>
        <version>1.2.0.Final</version>
        <scope>compile</scope>
    </dependency>

This works and no further downloads happen during the build step, which speeds up the build.

How can I cache Maven dependencies and plugins in a Docker Multi Stage Build Layer?

2 Answers2

Linked