How to loop parametrized parallel stages in Jenkins declarative pipeline?

Question

How to loop parametrized parallel stages in Jenkins declarative pipeline? (Or scripted pipeline, if declarative is not able to)

Here is my simple pseudo example. How to loop ('deploy serverN') stages?

Array may have 1..n variables.

I would not like to duplicate code. There must be a way in Jenkins pipelines?? Or should I use matrix. I have tried a few, but not succesfully.

@Library('adm-jenkins-lib@trunk')

def SERVERS = ['server1.com','server2.com',...]

deployPipeline([servers: SERVERS, manage_tasks: TASKS])

...
def call(Map params) {  
    pipeline {
        agent any
        environment {
        }
        stages {
            stage ('common task') {
            }       
            stage ('Deploying..') { 
                parallel {
                    stage ('deploy server1') {
                        stages {
                            stage ('deploy tasks #1') {
                                steps { ... }
                            }
                            stage ('deploy tasks #2') {
                                steps { ... }
                            }
                        }
                    stage ('deploy server2') {
                        stages {
                            stage ('deploy tasks #1') {
                                steps { ... }
                            }
                            stage ('deploy tasks #2') {
                                steps { ... }
                            }
                        }
                    }
                }
            }
        }
    }
}

I have tried also this kind of approach but it is not working perfect because previous stages are not dependent on next ones.

            stage ('deploy serverX') {
                when { expression { params.manage_tasks =~ /task01/ } }
                steps {
                    script {
                        servers = params.servers
                        servers.each { server ->
                            deploys[server] = {
                                sh "run task#1 stuff.."
                            }
                        }
                        parallel deploys
                    }
                }
            }

It should look like this in Blue Ocean (but dynamically created) : It should look like this in Blue Ocean

Does this answer your question? [Jenkins Pipeline - Run job on parallel on multiple remote hosts](https://stackoverflow.com/questions/60302232/jenkins-pipeline-run-job-on-parallel-on-multiple-remote-hosts) — zett42, Feb 21 '20 at 20:07
@zett42 Thanks for answer, but no, I have tried that solution and I can't use variables in Matrix axis values. It is expecting static values :( — Jari Ypyä, Feb 24 '20 at 08:19
Here is similar problem with Matrix axis values [link](https://stackoverflow.com/questions/59844846/declarative-pipeline-with-dynamic-matrix-axis-values) — Jari Ypyä, Feb 24 '20 at 10:55
OK, I didn't know that matrix is so static. I'm currently doing something similar to your 2nd approach in my project and it works quite well. What do you mean by "stages are not dependent on next ones"? — zett42, Feb 24 '20 at 18:36
You can create sub stages within the block `deploys[server] = { ... }` for "deploy tasks #n" and conditionalize them using [`imperative-when`](https://github.com/comquent/imperative-when) for nice display of skipped stages in blue ocean. This way I'm getting a result similar to your screenshot. — zett42, Feb 24 '20 at 18:45
Sounds interesting. Can you give an example, please? I have tried to add sub stages in script block, but while then it didn't work as expected.. — Jari Ypyä, Feb 25 '20 at 08:03
by "stages are not dependent on next ones" I meant stages should be conditional steps. Eg. when second stage fails, do not continue to stage 3 .. (all servers has own sequences of conditional stages) — Jari Ypyä, Feb 25 '20 at 10:00

Jari Ypyä · Accepted Answer · 2020-03-04T07:57:27.810

I have the solution:

Update Blue Ocean at least to version 1.22 to see pipeline correctly.

Install library https://github.com/comquent/imperative-when as @zett42 suggested.

This example is scripted pipeline. (I did not found solution for declarative pipeline)

@Library('adm-jenkins-lib@trunk') _

properties([
    parameters([
        string(name: 'countTotal', defaultValue: '4'),
        choice(name: 'servers', choices: ['all', '1', '2','3','4'], description: 'Run on specific platform'),
        choice(name: 'manage_steps', choices: ['all_tasks','common_task','deploy_task','test_task'], description: 'Choose task')
    ])
])

node{
    stage('common task'){
        when(params.manage_steps ==~ /common_task|all_tasks/) {
            sh "echo common task"   
        }
    }
    def stages = [failFast: true]
    for (int i = 1; i < params.countTotal.toInteger()+1; i++) {
        if (params.servers == 'all' || params.servers == i.toString() )
        {
            def vmNumber = i //alias the loop variable to refer it in the closure
            stages["server${vmNumber}"] = {
                stage("deploy ${vmNumber}") {
                        when(params.manage_steps ==~ /deploy_task|all_tasks/) {
                        sh "echo deploy; sleep 5"
                  }
                }
                stage("test ${vmNumber}") {
                        when(params.manage_steps ==~ /test_task|all_tasks/) {
                        sh "echo testing; sleep 5"
                        }
                }
            }
        }
    }
    parallel stages
}

BlueOcean Pipeline example1 BlueOcean Pipeline example2

How to loop parametrized parallel stages in Jenkins declarative pipeline?

1 Answers1

Linked