How to get CTest results in Hudson / Jenkins

Question

I'm using CTest (part of CMake) for my automated tests.

How do I get CTest results in the Jenkins dashboard ? Or, phrased differently, how do I get CTest to output in JUnit-like XML ?

Answer 1

In Jenkins, after the CMake part (probably made through the CMake plugin), add the following batch script, or adapt for builds on Linux :

del build_32\JUnitTestResults.xml
pushd build_32\Tests
"C:\Program Files\CMake 2.8\bin\ctest.exe" -T Test -C RelWithDebInfo --output-on-failure
popd
verify >nul
C:\Python27\python.exe external/tool/CTest2JUnit.py build_32/Tests external/tool/CTest2JUnit.xsl > build_32/JUnitTestResults.xml

• build_32 is the Build Directory in the CMake plugin
• Tests is the subdirectory where all my tests live
• -T Test makes CTest output in XML (?!)
• verify >nul resets errorlevel to 0, because CTest returns >0 if any test fails, which Jenkins interprets as "the whole build failed", which we don't want
• The last line converts CTest's XML into a minimal JUnit xml. The Python script and the xslt live in the source directory, you may want to change that.

The python script looks like this (hacked together in 10 min, beware) :

from lxml import etree
import StringIO
import sys

TAGfile = open(sys.argv[1]+"/Testing/TAG", 'r')
dirname = TAGfile.readline().strip()

xmlfile = open(sys.argv[1]+"/Testing/"+dirname+"/Test.xml", 'r')
xslfile = open(sys.argv[2], 'r')

xmlcontent = xmlfile.read()
xslcontent = xslfile.read()

xmldoc = etree.parse(StringIO.StringIO(xmlcontent))
xslt_root = etree.XML(xslcontent)
transform = etree.XSLT(xslt_root)

result_tree = transform(xmldoc)
print(result_tree)

• It needs lxml, direct link
• It takes two arguments, the directory in which the tests live (in the build directory), and a xsl file
• It simply reads the last xml tests results, transforms it with the xsl, and outputs it to stdout
• The "last xml tests" are present in the first line of the Testing/TAG file, hence the additional fopen

The xsl looks like this. It's pretty minimal but gets the job done : [EDIT] see MOnsDaR 's improved version : http://pastebin.com/3mQ2ZQfa

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="xml" indent="yes"/>

    <xsl:template match="/Site/Testing">    
        <testsuite>
            <xsl:apply-templates select="Test"/>
        </testsuite>
    </xsl:template>

    <xsl:template match="Test">
        <xsl:variable name="testcasename"><xsl:value-of select= "Name"/></xsl:variable>
        <xsl:variable name="testcaseclassname"><xsl:value-of select= "FullName"/></xsl:variable>
        <testcase name="{$testcasename}" classname="{$testcaseclassname}">
            <xsl:if test="@Status = 'passed'">
            </xsl:if>
            <xsl:if test="@Status = 'failed'">
                <error type="error"><xsl:value-of select="Results/Measurement/Value/text()" /></error>
            </xsl:if>
            <xsl:if test="@Status = 'notrun'">
                <skipped><xsl:value-of select="Results/Measurement/Value/text()" /></skipped>
            </xsl:if>
        </testcase>
    </xsl:template>

</xsl:stylesheet>

Finally, check "Publish JUnit tests results" (or similar, my version is in French) and set the xml path to build_32/JUnitTestResults.xml

Well, that was ugly. But still, hope this helps someone. And improvements are welcome ( running ctest from python maybe ? Using the path of the Python plugin instead of C:... ? )

Answer 2

This seems to be integrated in jenkins-ci nowadays:

• https://github.com/jenkinsci/xunit-plugin/commits/master/src/main/resources/org/jenkinsci/plugins/xunit/types/ctest-to-junit.xsl