Capture time taken by each pipeline stage in jenkins

Question

I am using a multibranch pipeline in jenkins and would like to record time taken by each pipeline stage and store it in DB. Any ideas on how to get the individual time durations for each build stage? I am just trying to get duration of each stage

Answer 1

One can use Jenkins workflow API as well to get the data of each stage -

https://Jenkins_URL/job/${jobName}/wfapi/runs

This will return an array of JSON objects (last 10 runs by default)

This data can then be stored in a time-series DB like InfluxDB.

Answer 2

The answer of Patrice M. seems to be the most elegant at first, but there is a problem with nested stages and branches. The duration of nested nodes won't be accounted for, instead only the node's own duration will be reported.

Fortunately we can access the TimingAction of the underlying FlowNode that is wrapped by FlowNodeWrapper to calculate the total duration, including nested nodes.

Solution

1. Find the FlowNode that represents the start of the branch or stage. Get the start time of this node by querying for its TimingAction.
2. Find the FlowNode that represents the end of the branch or stage. Get the start time of this node by querying for its TimingAction.
3. Calculate the difference between these two points in time.

Finding the nodes of branches and stages is quite easy using PipelineNodeGraphVisitor of Blue Ocean plugin API as I have shown in previous answers (for branches and for stages).

For getting the start time of a node, there is a handy static method TimingAction.getStartTime(node) which returns the time in milliseconds.

Example

import io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor
import io.jenkins.blueocean.rest.impl.pipeline.FlowNodeWrapper
import org.jenkinsci.plugins.workflow.actions.TimingAction

pipeline {
    agent any
    
    stages {
        stage('A') {
            stages {
                stage('A1') {
                    steps {
                        sleep 1
                    }
                }
                stage('A2') {
                    steps {
                        sleep 1
                    }
                }
            }
        }
    }
    post {
        always {
            printFinishedStageDurations()
        }
    }
}

void printFinishedStageDurations() {

    def visitor = new PipelineNodeGraphVisitor( currentBuild.rawBuild )

    // To find branches instead, replace NodeType.STAGE by NodeType.PARALLEL
    def stages = visitor.pipelineNodes.findAll{ it.type == FlowNodeWrapper.NodeType.STAGE }
    
    for( stage in stages ) {
        if( stage.node.endNode ) {   // only finished stages have endNode
            def startTime  = TimingAction.getStartTime( stage.node )
            def endTime    = TimingAction.getStartTime( stage.node.endNode )
            def duration   = endTime - startTime
        
            echo "Stage $stage.displayName duration: $duration ms" 
        }
    } 
}

Output

NOTE

When running the above example code in sandbox, you will get errors due to use of "unsafe" Jenkins API. This won't happen when moving printStageDurations() into a trusted shared library, as intended.

Answer 3

This has been open for some time, here is a possible answer based on @adir-d's excellent answer

import io.jenkins.blueocean.rest.impl.pipeline.*

def jobName = "folder/project/master"
def run = Jenkins.instance.getItemByFullName(jobName).getBuildByNumber(42)

PipelineNodeGraphVisitor visitor = new PipelineNodeGraphVisitor(run)
def stageNodes = visitor.getPipelineNodes().findAll { it.getType() == FlowNodeWrapper.NodeType.STAGE }

for (def node: stageNodes) {
    String stageName = node.getDisplayName()
    println "Result of stage ${node.getDisplayName()} is ${node.status.result} (${node.status.state})"
    println " timings: start=${node.timingInfo.startTimeMillis}ms; durationMillis=${node.timingInfo.totalDurationMillis}ms; paused=${node.timingInfo.pauseDurationMillis}ms"
    println "----------------------------------------------------------------"
}

Answer 4

Just for the record since this is unanswered for quite some time, afaik there is no builtin way to achieve this yet (maybe there is some plugin?), however the jenkins pipeline script functionality offers the flexibility to add this yourself, i.e. adding a commands that records a timestamp at the start of a staget and another one at the end of a stage.

I.e. check this regarding getting a timestamp value, you are free to use this for calculations and seding it to some database or other consumer.