How to have Maven artifacts persist across runs in Jenkins pipeline?

Question

We have a Maven Java project that we want to run in Jenkins BlueOcean pipelines.

I followed this tutorial. The pipeline is working to execute our code. Yay!

However, every time the 'build' stage of our pipeline executes under a new Jenkins run, it re-downloads all the maven artifacts. This increases our build time considerably.

I start the 'jenkins-docker' container with:

docker container run --name jenkins-docker --rm --detach \
  --privileged --network jenkins --network-alias docker \
  --env DOCKER_TLS_CERTDIR=/certs \
  --volume jenkins-docker-certs:/certs/client \
  --volume jenkins-data:/var/jenkins_home \
  --publish 2376:2376 docker:dind

And the 'jenkins-blueocean' container with:

docker container run --name jenkins-blueocean --rm --detach \
  --network jenkins --env DOCKER_HOST=tcp://docker:2376 \
  --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 \
  --volume jenkins-data:/var/jenkins_home \
  --volume jenkins-docker-certs:/certs/client:ro \
  --publish 8080:8080 --publish 50000:50000 jenkinsci/blueocean

Then our Jenkinsfile pipeline is:

pipeline {
    agent {
        docker {
            image 'maven:3.6.3-jdk-8' 
            args '-v /root/.m2:/root/.m2'
        }
    }
    stages {
        stage('Build') {
            steps {
                sh 'mvn -B -DskipTests clean package'
            }
        }
        stage('Test') { 
            steps {
                sh 'mvn test' 
            }
            post {
                always {
                    junit 'target/surefire-reports/*.xml' 
                }
            }
        }
    }
}

Here, Jenkins launches a new 'maven:3.6.3-jdk-8' docker image to do the run. It's also mapping a volume to persist the .m2 directory, as I understand.

Since my 'jenkins-docker' instance isn't shutting down across builds, I'd like to have this .m2 directory persisted. Then each successive run can leverage the cache of downloaded artifacts and not spend 5 minutes re-downloading them.

Is anyone able to offer any insight to what I'm doing wrong?

Thanks in advance

Answer 1

Finally figured it out.

The docker instance which runs the build doesn't automatically have a user.home variable. As a result, each time the build would execute, it'd store the .m2 artifacts under a path including a ? as one of the folders. Similar to what as described in this question.

From the Jenkins tutorial, I was able to figure out that the Maven Docker images expect to download artifacts to /root/.m2. This suggests the maven docker images use /root as their home directory.

I updated my Jenkinsfile to include this directive at the top so the Maven Docker instance would use it's de-facto home directory correctly.

environment {
  JAVA_TOOL_OPTIONS = '-Duser.home=/root'
}

Then I struggled with permissions issues trying to get the volume mapping right. Per the Jenkins setup, I already had created a volume jenkins-data, and it's available on my 'host' Jenkins containers as /var/jenkins_home.

Everything finally worked when I mapped this volume directly to the /root directory in my Maven docker image. This is a departure from the Jenkins tutorial setup, which showed this as being mapped to /root/.m2.

In summary

This is my working Jenkinsfile

    environment {
      JAVA_TOOL_OPTIONS = '-Duser.home=/root'
    }
    agent {
        docker {
            image 'maven:3.6.3-jdk-8' 
            args '-v /var/jenkins_home:/root'
        }
    }
    stages {
        stage('Build') {
            steps {
                sh 'mvn -B -DskipTests clean compile'
            }
        }
        stage('Test') { 
            steps {
                sh 'mvn -X test'
                jacoco()
            }
            post {
                always {
                    junit 'target/surefire-reports/*.xml' 
                }
            }
        }
    }

Answer 2

EDIT: i read the question as how to have production Jenkins maven builds avoid downloading artifacts by sharing the .m2 folder. The question is about getting a tutorial on running maven in a local docker instance to work. Not quite the same.

—-

Several solutions exist. Do not share/persist the .m2 folder, Maven is not thread safe (last time I really looked into this, might be fixed by now)

Simplest is to set up a Maven mirror repository (Nexus works well for this) and tell your docker Maven build to use that as a mirror.