Reuse agent (docker container) in Jenkins between multiple stages

Question

I have a pipeline with multiple stages, and I want to reuse a docker container between only "n" number of stages, rather than all of them:

pipeline {
   agent none

   stages {
       stage('Install deps') {
            agent {
                docker { image 'node:10-alpine' }
            }

            steps {
                sh 'npm install'
            }
        }

       stage('Build, test, lint, etc') {
            agent {
                docker { image 'node:10-alpine' }
            }

            parallel {
                stage('Build') {
                    agent {
                        docker { image 'node:10-alpine' }
                    }

                    // This fails because it runs in a new container, and the node_modules created during the first installation are gone at this point
                    // How do I reuse the same container created in the install dep step?
                    steps {
                        sh 'npm run build'
                    }
                }

                stage('Test') {
                    agent {
                        docker { image 'node:10-alpine' }
                    }

                    steps {
                        sh 'npm run test'
                    }
                }
            }
        }


        // Later on, there is a deployment stage which MUST deploy using a specific node,
        // which is why "agent: none" is used in the first place

   }
}

Oleksandr Shmyrko · Answer 1 · 2020-06-09T20:44:40.797

See reuseNode option for Jenkins Pipeline docker agent:
https://jenkins.io/doc/book/pipeline/syntax/#agent

pipeline {
  agent any

  stages {
    stage('NPM install') {
      agent {
        docker {
          /*
           * Reuse the workspace on the agent defined at top-level of
           * Pipeline, but run inside a container.
           */
          reuseNode true
          image 'node:12.16.1'
        }
      }

      environment {
        /*
         * Change HOME, because default is usually root dir, and
         * Jenkins user may not have write permissions in that dir.
         */
        HOME = "${WORKSPACE}"
      }

      steps {
        sh 'env | sort'
        sh 'npm install'
      }
    }
  } 
}

Exactly what I was looking for. Works perfectly in a declarative pipeline, thanks! — Bastien Jansen, Jul 21 '20 at 14:28

score 8 · Accepted Answer · answered May 21 '18 at 21:40

8

You can use scripted pipelines, where you can put multiple stage steps inside a docker step, e.g.

node {
  checkout scm
  docker.image('node:10-alpine').inside {
    stage('Build') {
       sh 'npm run build'
     }
     stage('Test') {
       sh 'npm run test'
     }
  }
}

(code untested)

answered May 21 '18 at 21:40

StephenKing

36,187
11
83
112

Yep, this worked. I realized after reading the docs more that the scripted pipeline is more powerful/ gives you better control of the pipeline. – Avindra Goolcharan May 25 '18 at 00:39
More specifically... the scripted pipeline uses imperative paradigms (whereas the declarative pipeline is directly opposed to this view). – Avindra Goolcharan Jul 29 '19 at 15:37

score 3 · Answer 3 · edited Jul 17 '23 at 19:18

3

For Declarative pipeline, one solution can be to use Dockerfile in the root of the project. For e.g.

Dockerfile

FROM node:10-alpine
// Further Instructions

Jenkinsfile

pipeline{

    agent {
        dockerfile true
    }
    stage('Build') {
        steps{
            sh 'npm run build'
        }
    }
     stage('Test') {
        steps{
            sh 'npm run test'
        }
    }
}

edited Jul 17 '23 at 19:18

Avindra Goolcharan

4,032
3
41
39

answered Apr 13 '19 at 22:29

Ankit Pandoh

548
5
14

Awesome. How do you checkout your repo first? (So there is a Dockerfile there to actually run. ...) – ingyhere May 08 '20 at 01:00
@ingyhere, Yes there is a Dockerfile to run. Please refer to my repo for example https://github.com/AnkitPandoh/infra-boiler . If you find it helpful, please upvote the answer. – Ankit Pandoh May 11 '20 at 00:36

score 2 · Answer 4 · edited Jul 17 '23 at 19:18

For declarative pipelines you can use nested stages described here

There's a better example than this in the link above, I just didn't want to copy paste verbatim.

pipeline {
    agent none

    stages {
        stage("Reuse Docker Image Across Sub-stages") {
            agent { docker "image1" }
            stages {
               stage("sub stage 1") {
                   steps { sh "./one.sh" }
               }
               stage("sub stage 2") {
                   steps { sh "./two.sh" }
               }
            }
        }

        stage("Use new docker image for something else") 
            agent { docker "image2" }
            steps { sh "./three.sh" }
        }
    }
}

Reuse agent (docker container) in Jenkins between multiple stages

4 Answers4

Linked