Set classname attribute for in TestResults.xml from NUnit Console Runner (workaround JENKINS-53349)

Question

I'm using NUnit and Jenkins and ran into Bug JENKINS-53349. It doesn't look like this will get fixed soon, so I would like to make this work.

This bug is annoying because you can't see which test failed and the test result output is not stable. Let's say you have a TestFixture with one parameter and you run this with 10 different values. The fixture has 5 tests. This will produce 50 test results from which only 5 are shown at random.

There is this Bug JENKINS-53349 which shows that the problem is the following:

In the TestResults.xml there is a <test-case> tag for each test case. This tag has a classname attribute. The linked Issues show that if this attribute reflects the parameterized class name the test-results-analyzer-plugin will render the results correctly.

<!-- "Bad" version -->
<test-case id="0-1001" name="TestMethod(1)" fullname="ClassLibrary1.TestClass(&quot;a&quot;).TestMethod(1)" methodname="TestMethod" classname="ClassLibrary1.TestClass" runstate="Runnable" seed="491809005" result="Passed" start-time="2018-08-30 13:07:59Z" end-time="2018-08-30 13:07:59Z" duration="0.008806" asserts="0"/>
<!-- "Fixed" version -->
<test-case id="0-1001" name="TestMethod(1)" fullname="ClassLibrary1.TestClass(&quot;a&quot;).TestMethod(1)" methodname="TestMethod" classname="ClassLibrary1.TestClass(&quot;a&quot;)" runstate="Runnable" seed="491809005" result="Passed" start-time="2018-08-30 13:07:59Z" end-time="2018-08-30 13:07:59Z" duration="0.008806" asserts="0"/>

Diff	attribute
Problem	`classname="ClassLibrary1.TestClass"`
Fixed	`classname="ClassLibrary1.TestClass("a")"`

I tried setting the TestName in the TestFixture, but that didn't work.

[TestFixture("a", TestName = "ClassLibrary1.TestClass(a)")]
[TestFixture("b", TestName = "ClassLibrary1.TestClass(b)")]
// …

Links

Doku of the XML Format from NUnit

Darkproduct · Accepted Answer · 2023-07-31T13:54:43.743

I wrote a small script in power shell to "fix" the XML.

param (
    [string]$xmlFilePath
)

# Check if the XML file exists
if (-not (Test-Path $xmlFilePath)) {
    Write-Host "Error: XML file not found at $xmlFilePath"
    Exit 1
}

# Load the XML file
[xml]$xmlDoc = Get-Content -Path $xmlFilePath

# Find all <test-case> elements in the XML
$testSuites = $xmlDoc.SelectNodes("//test-suite[@type='ParameterizedFixture']//test-case")

# Iterate over all elements and "fix" classname
# The real fix would be to fix jenkins
# - https://issues.jenkins.io/browse/JENKINS-53349
# - https://stackoverflow.com/questions/76045068
foreach ($testCase in $testSuites) {
    $i = $testCase.GetAttribute("fullname").LastIndexOf(".")
    $testCase.SetAttribute("classname", $testCase.GetAttribute("fullname").Substring(0, $i))
}

# $xmlDoc.Save([console]::out)

# Needs complete path
# https://stackoverflow.com/questions/4822575
$xmlDoc.Save((Resolve-Path $xmlFilePath))

Write-Host "XML file updated successfully"

Set classname attribute for in TestResults.xml from NUnit Console Runner (workaround JENKINS-53349)

Links

1 Answers1