How to force Jenkins to skip a downstream job if a new build is waiting in the pipeline?

Question

My build pipeline in Jenkins is broken down into three jobs:

Build code
Deploy code to environment
Run automated functional tests

I have set it up so that concurrent builds can occur and the build pipeline will stop a build from entering #2, if #2 or #3 is currently running for another build.

What I want to be able to do is set up Jenkins to handle when there is more than one build waiting, and #2 and #3 finish, for only the LATEST build to enter into #2 and #3.

Is there a way to do this out of the box? IF you have the book Continuous Delivery, what I'm trying to do is implement what's on p. 118 - p. 119

You could try looking at the top answer of http://stackoverflow.com/questions/12305244/cancel-queued-builds-and-aborting-executing-builds-using-groovy-for-jenkins — TomDotTom, Jul 22 '16 at 17:58

score 15 · Accepted Answer · edited May 23 '17 at 10:31

15

Should try one of those, under Advanced Project Options:

Block build when upstream project is building
(should make sure it does not cause steps 2 and 3 to get stuck in queue)
Block build when downstream project is building
(I know this one sounds like the opposite to your request,
but the actual result is that you accumulate changes to a single build-cycle,
preventing extra runs)

If this causes unwanted builds to pile-up,
please review the following links that should help you
empty the queue or kill running jobs:

Stopping Jenkins job in case newer one is started (by malenkiy_scot)
Purge Build Queue Plugin
Signal killer

Kill All the Builds (courtesy of Kohsuke Kawaguchi)

import hudson.model.*;

Hudson.instance.computers.each { c ->
  c.executors.each { e ->
     e.interrupt();
  } 
}

Cheers

edited May 23 '17 at 10:31

Community

1
1

answered Oct 11 '12 at 11:53

Gonen

4,005
1
31
39

I am currently using the "Block build when downstream project is building" in order to queue up the builds. The issue is that it queues up several builds and I have no say that interim builds (if there are more than one build in the queue) should do only #1 and skip the downstream steps of #2 and #3. Only the latest build in the queue should perform the downstream steps of #2 and #3. – BestPractices Oct 11 '12 at 16:42
I see... Added links that describe how to empty the queue or kill running builds. – Gonen Oct 12 '12 at 16:50

score 0 · Answer 2 · answered Sep 16 '15 at 01:59

0

The Workflow plugin lets you write your whole pipeline as one script. In this case the stage step can be used to control access:

stage 'build'
// any number of builds get here
stage name: 'deployAndTest', concurrency: 1
// only the last to build successfully enters here

(For visualization purposes, JENKINS-29892 would let you mark a boundary in between the deploy and test stages.)

answered Sep 16 '15 at 01:59

Jesse Glick

24,539
10
90
112

Does this work with the `input` step as well? Basically, "take the most recent success" with the `input` step? – mkobit May 13 '16 at 20:49
1

A `milestone` step is under development which should make that possible. – Jesse Glick May 14 '16 at 12:13

score 0 · Answer 3 · answered Apr 04 '16 at 20:38

The problem with "block upstream" or "block downstream" is that you're always blocking something which could be doing work.

If you use "git", you can do something along these lines - which happens to be what I am doing...

I use a tracking branch which points to the most recent finished build job of any step, named as follows: <branch>-latest-<step>. So, if you run a "build" step based on master, you would get master-latest-build. It is very easy to move this branch towards the end of your build script, just run: git branch -f <name> HEAD, followed by a push.

I then have the downstream jobs trigger off of that tracking branch. This way, all the jobs are loosely coupled and will do the right thing on whatever the upstream job produced, independent of what that upstream job is currently working on.

If, in addition to that, you also tag your build, your downstream job can retrieve the tag and re-use it as the build name to allow you to easily correlate various runs.

This is very effective if the pipeline has steps of vastly differing length - especially when your downstream steps take much longer than your upstream steps, which in my world is the norm, as the downstream tests include a whole suite of performance and integration tests...

How to force Jenkins to skip a downstream job if a new build is waiting in the pipeline?

3 Answers3

Linked