jenkinsfile pipeline grouping stages by agent

Question

What do I have

I am trying to run my jenkins pipeline using two different agents. I want to execute some process on the same agent but so far I am unable to do this because there are only 2 options for agent definition: I can do at top of pipeline or I can define the agent into each stage. I have this:

pipeline{
    agent none
    stages {
        stage("Unit Testing"){
            agent { label 'maven-build-slave' }
            steps{
            }
        }
        stage('Sonar Scanner - Quality Gates') {
            agent { label 'maven-build-slave' }
            steps{
            }
        }
        stage("Integration"){
            agent { label 'integration-slave' }
            steps{
            }
        }
        stage('SoapUI') {
            agent { label 'integration-slave' }
            steps{
            }
        }
    }
}

In this case the main problem is that the code is pulled in every stage even when the agent is the same.

What do I want

I would like something like this:

pipeline{
    agent none
    stages {
        agent { label 'maven-build-slave' }
        stage("Unit Testing"){
            steps{
            }
        }
        stage('Sonar Scanner - Quality Gates') {
            steps{
            }
        }

        agent { label 'integration-slave' }
        stage("Integration"){
            steps{
            }
        }
        stage('SoapUI') {
            steps{
            }
        }
    }
}

But the definition above is failing so I wonder if anyone knows a way to run several stages using same agent.

Answer 1

Check out the new (July 2018) Sequential Stages in Declarative Pipeline 1.3:

Running Multiple Stages with the Same agent, or environment, or options

While the sequential stages feature was originally driven by users wanting to have multiple stages in parallel branches, we’ve found that being able to group multiple stages together with the same agent, environment, when, etc has a lot of other uses.

For example, if you are using multiple agents in your Pipeline, but would like to be sure that stages using the same agent use the same workspace, you can use a parent stage with an agent directive on it, and then all the stages inside its stages directive will run on the same executor, in the same workspace.

pipeline {
    agent none

    stages {
        stage("build and test the project") {
            agent {
                docker "our-build-tools-image"
            }
            stages {
               stage("build") {
                   steps {
                       sh "./build.sh"
                   }
               }
               stage("test") {
                   steps {
                       sh "./test.sh"
                   }
               }
            }
            post {
                success {
                    stash name: "artifacts", includes: "artifacts/**/*"
                }
            }
        }

Answer 2

You may use this option as well. as stated in jenkins.io:

Short: if it is important to keep workspace synchronized with other stages, use reuseNode true. Otherwise, dockerized stage can be run on any other agent or on the same agent, but in temporary workspace.

By default, for containerized stage, Jenkins does:

pick any agent,

create new empty workspace,

clone pipeline code into it,

mount this new workspace into container.

If you have multiple Jenkins agents, your containerized stage can be started on any of them.

When reuseNode set to true: no new workspace will be created, and current workspace from current agent will be mounted into container, and container will be started at the same node, so whole data will be synchronized.

The key is using the reuseNode true inside the agent block inside each stage.