I guess my question is somewhat similar to Multiple JUnit XML results on Jenkins, publish with separate graph? but with another scope.
We need to test our software against multiple environments (e.g. windows/linux, different browsers, server environments). As far as I've seen this is a pretty common use case, but what I can't seem to find is how to have a clean report, where you can distinguish between these environments.
I tried the following simple pipeline:
node {
// dummy report
def testResultXml = '''
<testsuite name="ComponentTests" tests="3">
<testcase classname="foo1" name="ASuccessfulTest"/>
<testcase classname="foo2" name="AnotherSuccessfulTest"/>
<testcase classname="foo3" name="AFailingTest">
<failure type="NotEnoughFoo"> details about failure </failure>
</testcase>
</testsuite>
'''
stage('test run 1'){
writeFile file: 'test1.xml', text: testResultXml, encoding: 'UTF-8'
junit "test1.xml"
}
stage('test run 2'){
writeFile file: 'test2.xml', text: testResultXml, encoding: 'UTF-8'
junit "test2.xml"
}
}
and got a test report from Jenkins showing me the 2 failed tests. My problem is now, if only one fails I don't know which.
Our output is from a ant junit task that has the same output (except some properties) for all environments.
Even the article about declarative pipeline 1.2 shows multi-environment tests as a example: https://jenkins.io/blog/2017/09/25/declarative-1/ but says nothing about the presentation of the "Run Tests On Linux" and "Run Tests On Windows" results.
Did I miss something or is there a plugin that can publish the missing information (simply having the directory of the read junit xml file would help)?
