I've got a Jenkins server running onsite, and it uses a Jenkinsfile to manage a pipeline that uses the parallel test executor plugin to run all my JUnit tests on several agents to speed the test up. We have a blade server we made (way cheaper than buying one!) and it sped our tests from close to 2 hours to 22 minutes. The JUnit plugin works great with parallel tests.
The Jacoco Plugin however does not. So I am trying to get the coverage files merged to one file so that the Jacoco plugin can publish the coverage results. Stash/unstash is working in the storing of sources but it is not working when I try to stash the different Jacoco output files to unstash them on the master.
Any ideas why?
Here is my Jenkinsfile:
#!/usr/bin/env groovy
def branch
def hash
node('remote') {
sh 'echo starting'
branch = env.gitlabBranch ?: '**'
echo "Branch: $branch"
checkout([$class: 'GitSCM',
branches: [[name: "$branch"]],
extensions: [
[$class: 'PruneStaleBranch'],
[$class: 'CheckoutOption', timeout: 120],
[$class: 'CloneOption', depth: 0, noTags: true, shallow: true, timeout: 180]
],
doGenerateSubmoduleConfigurations: false,
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'gitlabLabptop', url: 'git@gitlab.com:protocase/my_project_url.git']]
]
)
hash = sh (script: 'git rev-parse HEAD', returnStdout: true).trim()
### - this stash works fine -###
stash name: 'sources', includes: '**', excludes: '**/.git,**/.git/**'
}
def numBranches = 9
def splits = splitTests count(numBranches)
def branches = [:]
for (int i = 0; i < splits.size(); i++) {
def index = i // fresh variable per iteration; i will be mutated
branches["split${i}"] = {
timeout(time: 125, unit: 'MINUTES') {
node('remote') {
sh 'echo starting a node'
deleteDir()
### - this unstash works fine - ###
unstash 'sources'
def exclusions = splits.get(index);
writeFile file: 'test/exclusions.txt', text: exclusions.join("\n")
sh 'ant clean'
sh 'rm -rf build'
sh 'ant jar'
sh 'ant -buildfile build-test.xml buildTests'
sh 'ant -buildfile build-test.xml jenkinsBatch'
junit 'build/test/results/*.xml'
sh "mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco${index}.exec"
echo "name: coverage$index, unclude jacoco${index}"
### - this stash appears to work - ###
stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
echo "stashed"
}
}
}
}
parallel branches
def branchIndecis = 0..numBranches
node('master') {
if (currentBuild.result != "ABORTED") {
echo "collecting exec files"
branchIndecis.each {
echo "unstash coverage${it}"
### !!! this unstash causes an error !!! ###
unstash name: "coverage${it}"
echo "make file name"
def coverageFileName = "build/test/jacoco/jacoco${it}.exec"
echo "merge file"
sh "ant -buildfile build-test.xml -Dfile=${coverageFileName} coverageMerge"
}
echo "collected exec files"
step([$class: 'JacocoPublisher',
execPattern:'build/test/jacoco/jacoco.exec',
classPattern: 'build/classes',
sourcePattern: 'src'])
echo "finishing ${branch} - ${hash}"
}
}
the output I get is:
[split7] [jdesigner] Running shell script
[split7] + mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco7.exec
[Pipeline] [split7] echo
[split7] name: coverage7, unclude jacoco7
[Pipeline] [split7] stash
[split7] Stashed 1 file(s)
[Pipeline] [split7] echo
[split7] stashed
[Pipeline] [split7] }
[Pipeline] [split7] // node
[Pipeline] [split7] }
[Pipeline] [split7] // timeout
[Pipeline] [split7] }
[Pipeline] // parallel
[Pipeline] node
Running on eightyeight in /var/jenkins/workspace/jdesigner
[Pipeline] {
[Pipeline] echo
collecting exec files
[Pipeline] echo
unstash coverage0
[Pipeline] unstash
[Pipeline] }
[Pipeline] End of Pipeline
Finished: FAILURE
[edit] the stash for coverage0 is
[split0] Recording test results
[Pipeline] [split0] sh
[split0] [jdesigner] Running shell script
[split0] + mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco0.exec
[Pipeline] [split0] echo
[split0] name: coverage0, include jacoco0
[Pipeline] [split0] stash
[split0] Stashed 1 file(s)
[Pipeline] [split0] echo
[split0] stashed
[Pipeline] [split0] }
[Pipeline] [split0] // node
[Pipeline] [split0] }
[Pipeline] [split0] // timeout
[Pipeline] [split0] }
[split3] [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.737 sec
[split3] [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.737 sec
note the line
[split0] name: coverage0, include jacoco0
is just my echo statement where I echo the name from this part of the script:
sh "mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco${index}.exec"
echo "name: coverage$index, include jacoco${index}"
stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
echo "stashed"
Note the actual stashing is not done on the node, it is listed as pipeline even though it is done on a remote node. I've seen things that indicated the stash is done on the master but not really where that directory resides.
[[FURTHER EDIT]] - thanks to eis for the recommendations.
The jobs/jdesigner/builds/1639/stashes/ directory on master has coverage#.tar.gz files that include the appropriate jacoco#.exec files. When I put a try catch around the unstash:
try {
unstash name: "coverage${it}"
} catch (error) {
echo "error unstashing: ${error}"
}
the output I get is:
collecting exec files
[Pipeline] echo
unstash coverage0
[Pipeline] unstash
[Pipeline] echo
error unstashing: java.io.NotSerializableException: groovy.lang.IntRange
[Pipeline] echo
make file name
