76

I use Jenkins and Multibranch Pipeline. I have a job for each active git branch. New build is triggered by push in git repository. What I want is to abort running builds in current branch if new one appears in same branch.

For example: I commit and push to branch feature1. Then BUILD_1 started in Jenkins. I make another commit and push to branch feature1 while BUILD_1 is still running. I want BUILD_1 to be aborted and to start BUILD_2.

I tried to use stage concurrency=x option, and stage-lock-milestone feature, but didn't manage to solve my problem.

Also I've read this thread Stopping Jenkins job in case newer one is started, but there is no solution for my problem.

Do you know any solution to this?

C4stor
  • 8,355
  • 6
  • 29
  • 47
kakty3
  • 1,149
  • 1
  • 9
  • 12
  • 1
    We let the current job finish, and them we have some cases where we let the jobs in queue be cleaned-up if we have never ones (as suggested in the referenced question.) Don't like the idea of aborting already started jobs. – MaTePe Nov 23 '16 at 10:01
  • 8
    @MaTePe For situations such as automated testing of git branches, there is frequently little benefit to completing a test on a branch if the branch has been updated as the updates will need to be tested as well. The obvious solution is to abort the earlier test. Cleanup may still need to be done, but resources aren't wasted completing an unnecessary test. – bschlueter Dec 05 '17 at 17:23

10 Answers10

52

With Jenkins script security many of the solutions here become difficult since they are using non-whitelisted methods.

With these milestone steps at the start of the Jenkinsfile, this is working for me:

def buildNumber = env.BUILD_NUMBER as int
if (buildNumber > 1) milestone(buildNumber - 1)
milestone(buildNumber)

The result here would be:

  • Build 1 runs and creates milestone 1
  • While build 1 is running, build 2 fires. It has milestone 1 and milestone 2. It passes milestone 1, which causes build #1 to abort.
  • 1
    Milestones are definitely the way to go with a multibranch declarative pipeline project. – Vladimir May 08 '19 at 06:27
  • [JENKINS-43353](https://issues.jenkins-ci.org/browse/JENKINS-43353) proposes making this official. – Jesse Glick Aug 01 '19 at 16:36
  • 2
    are milestones branch specific? – David Nov 19 '19 at 21:07
  • @David I cannot give you any documentation on this, but speaking from testing & experience - yes they are branch specific (the are not cancelling one another across branches at least not on my setup) – tymbark Mar 06 '20 at 15:39
  • @LucasCarnevalli that is true -- make sure the above milestone code is one of the first things defined in your Jenkinsfile. It does not require a 'node', so in theory you should be able to run this code before anything else runs. If your job is failing this early in the job due to a failed import or something like that you probably have bigger problems to sort out :) – Brandon Squizzato Jul 01 '20 at 18:30
  • @brandonsquizzato - where does this go in the Jenkinsfile? – Alex G Oct 13 '20 at 17:14
  • Super cool feature. Thanks for this. Also works with scripted pipeline. – Michael S Dec 11 '20 at 16:13
  • What is the reason for the line if(buildNum>1) milestone(buildNum - 1)? Wouldn't jus the milestone(buildNum) suffice ? – silvio Dec 14 '22 at 16:24
31

enable job parallel run for your project with Execute concurrent builds if necessary

use execute system groovy script as a first build step:

import hudson.model.Result
import jenkins.model.CauseOfInterruption

//iterate through current project runs
build.getProject()._getRuns().iterator().each{ run ->
  def exec = run.getExecutor()
  //if the run is not a current build and it has executor (running) then stop it
  if( run!=build && exec!=null ){
    //prepare the cause of interruption
    def cause = { "interrupted by build #${build.getId()}" as String } as CauseOfInterruption 
    exec.interrupt(Result.ABORTED, cause)
  }
}

and in the interrupted job(s) there will be a log:

Build was aborted
interrupted by build #12
Finished: ABORTED 
daggett
  • 26,404
  • 3
  • 40
  • 56
  • 1
    Sounds very good ! Currently looking for a way to port it to a pipeline file scm commited – C4stor Jun 02 '17 at 15:20
  • The `system groovy script` runs inside the Jenkins master's JVM, that's why it can access everything in jenkins. But pipeline runs in a forked JVM, on the slave where the build is run - I did not find it in docs, but quite sure it is. – daggett Jun 02 '17 at 20:23
  • i've found this: https://stackoverflow.com/questions/33531868/jenkins-workflow-build-information / i have no pipeline to try now but you can try to get current build in pipeline using this expression: `def build = currentBuild.rawBuild` – daggett Jun 03 '17 at 15:37
  • 1
    Got the code to work, but curiously, _getRuns only ever list the current running build :/ – C4stor Jun 05 '17 at 08:29
  • what is the value of `currentBuild.rawBuild.getClass()` ? – daggett Jun 05 '17 at 18:19
  • 1
    class org.jenkinsci.plugins.workflow.job.WorkflowRun – C4stor Jun 06 '17 at 07:31
  • Thanks to that question I looked at the correct javadoc and got a working solution ^^ – C4stor Jun 06 '17 at 07:46
  • 2
    For anybody who, like me, got to this answer and has trouble making the code run - remove the id from the closure. basically change line: `build.getProject()._getRuns().each{id,run-> ` into `build.getProject()._getRuns().each{ run ->` – kars7e Jul 28 '17 at 09:12
  • It gives me this error even after defining build. groovy.lang.MissingPropertyException: No such property: build for class: groovy.lang.Binding at groovy.lang.Binding.getVariable(Binding.java:63) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:242) – Taha Tariq Dec 05 '17 at 11:07
  • 1
    it will not work in sandbox. `execute system groovy script` – daggett Dec 05 '17 at 12:21
  • 1
    Also this approach would seem to be risky in regards to master branch. If you work in a team, you merged to master, your team mate merged 1 min later. your job would be canceled and his would run. – JarJarrr Mar 06 '18 at 15:15
  • system groovy seems to be disabled in recent jenkins releases..... i dont think this is still possible – U.V. Feb 14 '19 at 13:59
  • @daggett What is meant with 'will not work in sandbox'? – Michael Kemmerzell Mar 06 '19 at 07:13
  • I was not able to run it with "script sandbox security": https://wiki.jenkins.io/display/JENKINS/Script+Security+Plugin – daggett Mar 06 '19 at 07:29
  • 1
    I'm not sure this will work unless the build is being run on an executor. In my case, I need this to trigger when the build is QUEUED. Is that possible? This is because we have so many parallel PRs that are rebuilt upon a merge into Develop, that many are queued and stay queued because stale branches are being built – Mark Han Oct 01 '19 at 16:33
  • remove queue and make builds run in parallel, so last build will stop the previous. – daggett Oct 01 '19 at 16:43
  • Will that cancel downstream projects as well? – user972014 Jul 10 '20 at 05:39
  • For anyone else, we had to `previousBuild.getListener().getLogger().println("abort reason")` rather than specify custom cause to prevent builds from resuming on a restart – Elijah Jan 14 '22 at 17:18
22

If anybody needs it in Jenkins Pipeline Multibranch, it can be done in Jenkinsfile like this:

def abortPreviousRunningBuilds() {
  def hi = Hudson.instance
  def pname = env.JOB_NAME.split('/')[0]

  hi.getItem(pname).getItem(env.JOB_BASE_NAME).getBuilds().each{ build ->
    def exec = build.getExecutor()

    if (build.number != currentBuild.number && exec != null) {
      exec.interrupt(
        Result.ABORTED,
        new CauseOfInterruption.UserInterruption(
          "Aborted by #${currentBuild.number}"
        )
      )
      println("Aborted previous running build #${build.number}")
    } else {
      println("Build is not running or is current build, not aborting - #${build.number}")
    }
  }
}
midN
  • 567
  • 5
  • 8
  • 2
    Maybe it is worth checking that the build number is lower than the current. Otherwise, you might kill even newer builds. – danielMitD Feb 16 '18 at 10:13
22

From Jenkins workflow-job plugin version 2.42 you can simply do

// as a step in a scripted pipeline    
properties([disableConcurrentBuilds(abortPrevious: true)]) 
// as a directive in a declarative pipeline
options { disableConcurrentBuilds abortPrevious: true } 

Found solution in comments here https://issues.jenkins.io/browse/JENKINS-43353

Maxence Guichon
  • 221
  • 2
  • 2
  • Sorry, but is that release even out yet? I find the latest to be `2.327`... – Johny Serpa Jan 12 '22 at 12:20
  • 1
    Got it, you're talking about the plugin https://plugins.jenkins.io/workflow-job/#releases – Johny Serpa Jan 12 '22 at 12:33
  • 1
    42 < 327 this trips me up all the time. It should be 042. – Max Cascone Jun 16 '22 at 18:34
  • Does this work to allow concurrent builds across branches/PRs but terminate old ongoing build jobs for a given branch/PR? – Steven Lu Sep 09 '22 at 04:48
  • The answer is perfectly working. But in my case, the old build takes some time to abort, So the new build automatically takes a second workplace i.e "xxxxxx@2". since I am hardcoding the workspace path so the new build is getting failed. is there any workaround for this? – Ojer_Dev Nov 23 '22 at 19:05
  • works like a charm... simple and straight forward :) – P. Sithole Nov 24 '22 at 07:27
14

Based on the idea by @C4stor I have made this improved version... I find it more readable from @daggett 's version

import hudson.model.Result
import hudson.model.Run
import jenkins.model.CauseOfInterruption.UserInterruption

def abortPreviousBuilds() {
    Run previousBuild = currentBuild.rawBuild.getPreviousBuildInProgress()

    while (previousBuild != null) {
        if (previousBuild.isInProgress()) {
            def executor = previousBuild.getExecutor()
            if (executor != null) {
                echo ">> Aborting older build #${previousBuild.number}"
                executor.interrupt(Result.ABORTED, new UserInterruption(
                    "Aborted by newer build #${currentBuild.number}"
                ))
            }
        }

        previousBuild = previousBuild.getPreviousBuildInProgress()
    }
}
Stefanos Kalantzis
  • 1,619
  • 15
  • 23
  • This solved the problem in my pipeline script. The "Aborting older build" message is being displayed, but the "Aborted by newer build" isn't. Maybe it is because my older build was waiting for a input action. – neves Jan 16 '19 at 17:38
  • @neves Could be. Also just in case it's not obvious: the "Aborted by newer build" message is displayed on the other (older) build. – Stefanos Kalantzis Jan 22 '19 at 11:02
  • This approach is using static methods. So I'm geting this error: Scripts not permitted to use staticMethod hudson.model.Hudson getInstance – Dmitry Kuzmenko Jul 22 '21 at 19:32
  • 1
    @DmitryKuzmenko Maybe you're running the script inside the sandbox? It wouldn't work there. Also, this is from 2018, maybe there are differences in newer versions. – Stefanos Kalantzis Aug 11 '21 at 19:31
  • Where does "currentBuild" come from? We have a lot of custom code I'm struggling to parse in our jenkins setup and I'm not sure what type that object is. – Yamikuronue Feb 11 '22 at 15:04
9

Got it to work by having the following script in the Global Shared Library :

import hudson.model.Result
import jenkins.model.CauseOfInterruption.UserInterruption

def killOldBuilds() {
  while(currentBuild.rawBuild.getPreviousBuildInProgress() != null) {
    currentBuild.rawBuild.getPreviousBuildInProgress().doKill()
  }
}

And calling it in my pipeline :

@Library('librayName')
def pipeline = new killOldBuilds()
[...] 
stage 'purge'
pipeline.killOldBuilds()

Edit : Depending on how strongly you want to kill the oldBuild, you can use doStop(), doTerm() or doKill() !

C4stor
  • 8,355
  • 6
  • 29
  • 47
  • Is there any way to send a message to the terminated build?It sends this hard kill signal but no log on who killed it. – Taha Tariq Dec 04 '17 at 16:21
  • I wouldn't know, we're living with this full gray lines for the moment, good enough for us ^^' – C4stor Dec 05 '17 at 14:11
  • 1
    The order from graceful to destructive goes `doStop()` -> `doTerm()` -> `doKill()` – Deiwin Apr 12 '18 at 16:11
  • How did this ever work for you? it's wrong :) But thanks for the idea... I have a working version... see my answer – Stefanos Kalantzis Apr 18 '18 at 13:45
  • Well, it's working in our production stack right now, so I don't think it's wrong. The fact you were unable to use the code as it can come from a lot factors, including jenkins version, java version, os used, file permissions in use.... – C4stor Apr 18 '18 at 16:04
  • @TahaTariq the solution https://stackoverflow.com/a/49901413/10335 prints a message in older jobs. Unfortunately it looks it doesn't work if your other running job is waiting for a input. – neves Jan 16 '19 at 17:40
5

Adding to Brandon Squizzato's answer. If builds are sometimes skipped the milestone mechanism as mentioned will fail. Setting older milestones in a for-loop solves this.

Also make sure you don't have disableConcurrentBuilds in your options. Otherwise the pipeline doesn't get to the milestone step and this won't work.

def buildNumber = env.BUILD_NUMBER as int
for (int i = 1; i < buildNumber; i++)
{
    milestone(i)
}
milestone(buildNumber)
CodeMonkey
  • 4,067
  • 1
  • 31
  • 43
  • 3
    The potential problem with this is that when you have a large number of builds, creating that many milestones can take up a considerable amount of time. I don't know exactly at what point things changed -- creating many milestones used to go for quickly for me. Then more recently, creating a milestone was taking about half a second each -- obviously not ideal if you are on build #900. So I created my solution that does not use a for loop. – Brandon Squizzato Jul 02 '20 at 21:18
1

Based on @daggett method. If you want to abort running build when the new push is coming and before fetch updates.
1. Enable Execute concurrent builds if necessary
2. Enable Prepare an environment for the run
3. Running bellow code in Groovy Script or Evaluated Groovy script

import hudson.model.Result
import hudson.model.Run
import jenkins.model.CauseOfInterruption

//def abortPreviousBuilds() {
    Run previousBuild = currentBuild.getPreviousBuildInProgress()

    while (previousBuild != null) {
        if (previousBuild.isInProgress()) {
            def executor = previousBuild.getExecutor()
            if (executor != null) {
                println ">> Aborting older build #${previousBuild.number}"
                def cause = { "interrupted by build #${currentBuild.getId()}" as String } as CauseOfInterruption 
                executor.interrupt(Result.ABORTED, cause)
            }
        }
        previousBuild = previousBuild.getPreviousBuildInProgress()
    }
//}
Ian Kwok
  • 11
  • 2
1

Before each build task, first determine whether all the tasks currently under construction are the same as the branch of this build. If they are the same, keep the latest task.

    stage('Setup') {
        steps {
            script {
                JOB_NAME = env.JOB_NAME
                branch_name = "${env.gitlabTargetBranch}"
                def job = Jenkins.instance.getItemByFullName( JOB_NAME )
                def builds = job.getBuilds()
                for( build in job.getBuilds()) {
                    if (build.isBuilding()) {
                      String parameters = build?.actions.find{ it instanceof ParametersAction }?.parameters?.collectEntries {
                            [ it.name, it.value ]
                            }.collect { k, v -> "${v}" }.join('\n')
                            
                             if (branch_name == "${parameters}") {
                                 if (env.BUILD_NUMBER > "${build.getId()}") {
                                     build.doKill()  
                                 }
                             }
                    }
                }
                
                
            }
           ......
0

I also compiled a version from the previously given ones with a few minor tweaks:

  • the while() loop generated multiple outputs for each build
  • the UserInterruption currently expects a userId instead of a reasoning string, and will not show a reasoning string anywhere. Therefore this just provides the userId
def killOldBuilds(userAborting) {
    def killedBuilds = []
    while(currentBuild.rawBuild.getPreviousBuildInProgress() != null) {
        def build = currentBuild.rawBuild.getPreviousBuildInProgress()
        def exec = build.getExecutor()

        if (build.number != currentBuild.number && exec != null && !killedBuilds.contains(build.number)) {
            exec.interrupt(
                    Result.ABORTED,
                    // The line below actually requires a userId, and doesn't output this text anywhere
                    new CauseOfInterruption.UserInterruption(
                            "${userAborting}"
                    )
            )
            println("Aborted previous running build #${build.number}")
            killedBuilds.add(build.number)
        }
    }
}
Janis Peisenieks
  • 4,938
  • 10
  • 55
  • 85