How do I stop Jenkins from stalling when an agent is turned off?

Question

My Jenkins agents are normal workstations that can be turned off. If this happens during a job, I expect Jenkins to fail that job. Instead, the job stalls indefinitely, even after the agent comes back. How do I make it fail instead? Restarting from the last step would be even better, but that is very much a nice to have.

The last log message is

Cannot contact someagent: java.lang.InterruptedException

When I abort the stallling job, I get

Aborted by admin
Sending interrupt signal to process
Click here to forcibly terminate running steps
After 20s process did not stop

What process did not stop? Nothing can possibly be running on the agent at this point.

Jenkins version is latest LTS, the job is a declarative pipeline that invokes PowerShell scripts. An excerpt:

stage('Build')
{
    steps
    {
        powershell '. Tools\\Build\\Build.ps1 -version $env:VERSION'
    }
}
stage('Package')
{
    steps
    {
        powershell '. Tools\\Build\\Package.ps1 -artifact $env:ARTIFACT'
        archiveArtifacts ARTIFACT
    }
}

Robin Fauvel · Answer 1 · 2023-07-13T09:12:09.903

The Jenkins Plugin Pipeline: Basic Steps offers a step called timeout that might interest you.

It can be applied to a single stage, or to the whole pipeline.

Here is an example using the Scripted Syntax:

timeout(time: 30, unit: 'MINUTES) {
    node {
      stage('test') { ... }
    }
}

If you are using the Declarative Syntax, the timeout option declared at the top-level agent only takes effect after the agent becomes available, which is not what you are trying to achieve. Instead, you can workaround it by using agent none at the top level and specify your agent for each stage. This phenomenon is actually described in the documentation.

Example:

pipeline {
    agent none
    options {
        timeout(time: 1, unit: 'HOURS') 
    }
    
    stages {
        stage('Hello') {
            agent { label 'my-label' }
            steps {
                echo 'Hello World'
            }
        }
    }
}

See also this other thread: How to add a timeout step to Jenkins Pipeline

How do I stop Jenkins from stalling when an agent is turned off?

1 Answers1