29

I want to define multiple stages in Jenkins declarative pipeline syntax which can continue past any one of them failing. I cannot find any existing questions which are true duplicates, because they all assume or allow scripted syntax.

pipeline {
    agent any
    stages {
        stage('stage 1') {
            steps {
                echo "I need to run every time"
            }
        }
        stage('stage 2') {
            steps {
                echo "I need to run every time, even if stage 1 fails"
            }
        }
        stage('stage 3') {
            steps {
                echo "Bonus points if the solution is robust enough to allow me to continue *or* be halted based on previous stage status"
            }
        }
    }
}

To clarify, I'm not looking for how to do accomplish this in scripted syntax. I'm trying to understand if this kind of flow control is actually supported and formalized in declarative syntax. To that end, I'll try to define exactly what I'm looking for:

Required

  • No try/catch. I don't want to drop down into scripted mode, or "wrap" my declarative pipeline in another shared library or scripted block.
  • No post step shenanigans. I want true multiple stages, not one stage with a post always step that contains all my other logic

Optional

  • The failing stage should be recognized as failed; I don't want a failed stage showing up as green because it was "skipped" or "continued".
  • A build with any failed stage should be marked as red (or yellow, or anything that is not green).

Related but Not Sufficient

StephenKing
  • 36,187
  • 11
  • 83
  • 112
dolphy
  • 6,218
  • 4
  • 24
  • 32
  • 2
    I think some of your requirements are mutually exclusive from a conceptual POV. 1) A pipe(line) has two openings. If it breaks nothing will go past the breakpoint unless you do some plumbing, e.g. `try/catch`. 2) Thinking of Maven with its declarative POMs, if one phase (stage) fails there the whole build fails, with no chance to overcome this but to remove the cause and try again. [to be continued] – Gerold Broser Jul 11 '17 at 03:14
  • [cont'd] 3) If one looks at the example of the [Pipeline Stage View Plugin](https://wiki.jenkins.io/display/JENKINS/Pipeline+Stage+View+Plugin) with _Build_ → _Deploy_ → _Test_ → _Promote_ stages it doesn't make sense to continue if one stage fails. – Gerold Broser Jul 11 '17 at 03:15
  • 1
    While I do absolutely understand that point of view, I don't agree with it personally. To me, flow control of a pipeline is an absolutely essential part of the construct...it's just to restrictive to be very useful otherwise. Even the existence of try/catch in scripted syntax seems to be a nod to that; my question is how that element of flow control is formalized in declarative. Maybe the answer is "it isn't yet", I don't know. – dolphy Jul 11 '17 at 14:55
  • I think "(not) yet" _is_ the keyword in this respect, since the [Pipeline Syntax reference](https://jenkins.io/doc/book/pipeline/syntax/#declarative-pipeline) says: "_Declarative Pipeline is a relatively recent addition to Jenkins Pipeline [1] which presents a more simplified and opinionated syntax on top of the Pipeline sub-systems. [...] 1. Version 2.5 of the "Pipeline plugin" introduces support for Declarative Pipeline syntax_" – Gerold Broser Jul 12 '17 at 23:37

4 Answers4

29

This is now possible:

pipeline {
    agent any
    stages {
        stage('1') {
            steps {
                sh 'exit 0'
            }
        }
        stage('2') {
            steps {
                catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
                    sh "exit 1"
                }
            }
        }
        stage('3') {
            steps {
                sh 'exit 0'
            }
        }
    }
}

In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:

Pipeline Example

As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.

Just make sure your Jenkins is up to date, since this is a fairly new feature.

EDIT: This is the question that this answer was originally written for. It is also the correct answer for a few other questions, which is why I posted this answer there as well. This is the right solution for multiple similar problems. I've tailored my other answers to their specific questions to make that clear. I only copied the answer to save myself some time. That doesn't mean it's not a good correct answer.

Erik B
  • 40,889
  • 25
  • 119
  • 135
4

I may be missing something, but the idea of declarative, opinionated pipeline is to provide coverage of most simple use cases. The moment you need something the opinionated hasn't covered, you HAVE to resort to scripted pipeline, this is only referring to the "requirement" of "declarative pipeline": not going to happen now.

As to your other "requirements", they make very little sense, since the whole idea is to wrap low-level ugliness into shared libraries providing users with constructs like:

    mylib.failable_stages({
      stages {
        stage('stage 1') {
          steps {
            echo "I need to run every time"
          }
        }
        stage('stage 2') {
          steps {
            echo "I need to run every time, even if stage 1 fails"
          }
        }
        stage('stage 3') {
          steps {
            echo "Bonus points if the solution is robust enough to allow me to continue *or* be halted based on previous stage status"
          }
        }
      }
    })

Naturally, you would have to find or implement such mylib class and the failable_stages would get a closure, and wrap it in various plumbing/boilerplate code pieces.

Hope this is helpful.

mvk_il
  • 940
  • 8
  • 11
  • Thanks for the reply. The technical response is exactly why I worded the question the way that I did. I'm trying to understand whether or not this kind of flow control is formalized in the declarative syntax. Any time anything close to this question is asked the response is usually "here's how you do it in scripted". That's a solution, but it's not an answer to the question. It's kind of like me asking "How do I get a directory listing in Python", and being told to "just use subprocess.check_output(["ls"])". It's a solution, but not the answer to the question. – dolphy Jul 11 '17 at 15:22
  • 2
    I will respectfully disagree about the purpose of shared libraries. My understanding is that they are intended to abstract sharable code, not ugly code. If code is put in a shared library just to hide it, without the intention of reusing it in multiple places, then that is a failing of the language or the developer or both. Maybe this is just a case of just two different view points, and maybe the answer to my question really is "You can't do it", but I do truly want to know that. I'm asking the question because I know I'm not the smartest guy here. – dolphy Jul 11 '17 at 15:24
  • In any case, the fact that your interpretation of the question is not what I wanted to present means I need to reword something. I've added a clarification, let me know if that helps or not! – dolphy Jul 11 '17 at 15:34
  • Hi, @dolphy. I'm better now. Removed the "Ethical" paragraph. Still not clear why you're stuck on being declarative. Also there is another path: 1. Hackish: run your stages inside single item'ed `parallel` steps. They have a parameter to not fail build if the internal steps fail. 2. True solution would be to extend `stage` by adding it an extra boolean paramet – mvk_il Jul 14 '17 at 08:26
  • @dolphy my bad phrasing this time. It so happens to be that the code you share a lot, without the abstraction makes your job groovy longer. It in itself is ugly. Besides the 'wrappers' make the code look nested, which is also ugly. So even if we discard beauty pure logic and structure, visual appeal and readability is worse without shared libraries. IMHO, of course. – mvk_il Jul 14 '17 at 08:36
  • @dolphy, forgot to mention that my customers usually already have some freestyle jobs. So making them learn groovy is not always met with utmost excitement of the employees. So making them maintain short jobs is a big + – mvk_il Jul 14 '17 at 08:39
2

I think it depends on how dependent the jobs are on each other. Derived from your example I would assume that

  • stage 1 is independent from all other stages, because it is the first
  • stage 2 is independent from all other stages, because stage 1 might fail immediatly and stage 2 would still be required to run
  • stage 3 depends on the result of stage 1 and stage 2

So the correpsonding pipeline could be

pipeline {
    stages {
        stage('Independent tasks') {
            parallel {
                stage('stage 1') {
                    steps {
                        sh 'exit 1' // failure
                    }
                }
                stage('stage 2') {
                    steps {
                        echo 'Happens even so stage 1 fails'
                        sh 'exit 0' // success
                    }
                }
            }
            post {  // 'stage 3'
                failure {
                    echo "... at least one failed"
                }
                success {
                    echo "Success!"
                }
            }
        }
        stage ('stage 4') {
            steps {
                echo 'Happens only if all previous succeed'
            }
        }
    }
}

stage 1 and stage 2 will always run, stage 3 reacts on their combined success/failure.


Additional thought: This concept only works 'at the end' of your pipeline. In case you need it somewhere in the middle AND the build has to continue, you could move it into an own job and use the build job plugin.

pipeline {
    stages {
    stage('Start own job for stage 1, 2, 3') {
        steps {
            build job: 'stageOneTwoThree', propagate: false, wait: true
        }
    }
    stage ('stage 4') {
        steps {
            echo 'Happens always, because "propagate: false"'
        }
    }
}
newur
  • 490
  • 4
  • 12
2

I achieved it using post. My requirement was to send out slack notification irrespective of the build status.

@Library('instanceGroups')
import x.z.y.jenkins.libraries.SlackNotifier

def slackHelper = new x.z.y.jenkins.libraries.SlackNotifier(env)
final String projectName = "pomeranian"
final String featureBranchPattern = "f_"

pipeline {
    agent any
    options { disableConcurrentBuilds() }

    stages {
        stage('clean') {
            steps {
                script {
                    try {
                        echo 'Current Branch...' + env.BRANCH_NAME
                        sh 'rm -rf /var/lib/jenkins/.gradle/caches'
                        sh './gradlew clean'
                    } catch (e) {
                        currentBuild.result = 'FAILURE'
                        slackHelper.buildGenericJobFailureNotificationMessage()
                        throw e
                    }
                }
            }
        }

        stage('compile') {
            steps {
                script {
                    try {
                        sh "./gradlew compileJava"
                    } catch (e) {
                        currentBuild.result = 'FAILURE'
                        slackHelper.getCompilationFailureSlackNotificationMessage()
                        throw e
                    }
                }
            }
        }

        stage('test') {
            steps {
                script {
                    try {
                        sh "./gradlew test"
                    } finally {
                        junit 'build/test-results/test/*.xml'

                        slackHelper.getTestStatuses(currentBuild)
                        slackHelper.buildUnitTestSlackNotificationMessage()
                    }
                }
            }
        }

        stage('publish 2 nexus') {
            steps {
                script {
                  // some code
                }
            }
        }

        stage('git tagging') {
            steps {
                script {
                    // some more code...
            }
        }
    }


    post {
        always {
            script {
                slackHelper.finalBuildStatusMessage(currentBuild)
                slackSend(channel: '#ci-cd', attachments: slackHelper.getFinalSlackMessage())
            }
        }
    }
}
smaikap
  • 474
  • 1
  • 7
  • 19