Anyone have an XSL to convert Boost.Test XML logs to a presentable format?

Question

I have some C++ projects running through cruisecontrol.net. As a part of the build process, we compile and run Boost.Test unit test suites. I have these configured to dump XML log files. While the format is similar to JUnit/NUnit, it's not quite the same (and lacks some information), so cruisecontrol.net is unable to pick them up. I am wondering if anyone has created (or knows of) an existing XSL transform that will convert Boost.Test results to JUnit/NUnit format, or alternatively, directly to a presentable (html) format.

Thanks!

JUnit is since boost 1.62 a natively supported format for boost.test — Raffi, Apr 20 '17 at 17:39

Stuart Lange · Accepted Answer · 2010-06-04T20:08:02.177

I'm working on rolling my own Boost.Test -> JUnit XSL. Please note that this is intended to consume the XML report output from Boost.Test - not the log output. This is a work in progress - here's what I have so far:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:msxsl="urn:schemas-microsoft-com:xslt"
                exclude-result-prefixes="msxsl">

  <xsl:output method="xml"
              indent="yes"/>

  <xsl:template match="TestResult">
    <test-results>
      <xsl:attribute name="total">
        <xsl:value-of select="sum(./TestSuite/@test_cases_passed) + sum(./TestSuite/@test_cases_failed) + sum(./TestSuite/@test_cases_skipped) + sum(./TestSuite/@test_cases_aborted)"/>
      </xsl:attribute>
      <xsl:attribute name="failures">
        <xsl:value-of select="sum(./TestSuite/@test_cases_failed) + sum(./TestSuite/@test_cases_aborted)"/>
      </xsl:attribute>
      <xsl:attribute name="skipped">
        <xsl:value-of select="sum(./TestSuite/@test_cases_skipped)"/>
      </xsl:attribute>
      <xsl:attribute name="not-run">
        <xsl:value-of select="sum(./TestSuite/@test_cases_skipped)"/>
      </xsl:attribute>
      <xsl:call-template name="testSuite" />
    </test-results>
  </xsl:template>

  <xsl:template name="testSuite">
    <xsl:for-each select="TestSuite">
      <test-suite>
        <xsl:call-template name="testAttributes" />
        <results>
          <xsl:call-template name="testSuite" />
          <xsl:for-each select="TestCase">
            <test-case>
              <xsl:call-template name="testAttributes" />
            </test-case>
          </xsl:for-each>
        </results>
      </test-suite>
    </xsl:for-each>
  </xsl:template>

  <xsl:template name="testAttributes">
    <xsl:attribute name="name">
      <xsl:value-of select="@name"/>
    </xsl:attribute>
    <xsl:attribute name="success">
      <xsl:choose>
        <xsl:when test="@result = 'passed'">True</xsl:when>
        <xsl:when test="@result != 'passed'">False</xsl:when>
      </xsl:choose>
    </xsl:attribute>
    <xsl:attribute name="executed">True</xsl:attribute>
    <xsl:attribute name="time">0</xsl:attribute>
    <xsl:attribute name="asserts">
      <xsl:value-of select="@assertions_failed + @assertions_passed"/>
    </xsl:attribute>
  </xsl:template>

</xsl:stylesheet>

I have this integrated into my build process and it's getting picked up and processed by ccnet nicely. It's not perfect, but it works better than the complete lack of reporting I had before. I'm open to suggestions on how to map the Boost.Test data to the "total", "failures", "skipped", and "not-run" fields of the JUnit report. Also, unfortunately the error detail data (indicating the nature of the failure and the file/line number where the failure occurred) are only printed to the log, not to the report, so I would have to "merge" the two to get all the data I would ideally like to have.

score 5 · Answer 2 · answered Jan 24 '11 at 18:49

Thanks to Stuart Lange's answer above, I was able to get Boost.Test's output to integrate with our Bamboo build system. Bamboo describes the format they can ingest here:

http://confluence.atlassian.com/display/BAMBOO/JUnit+parsing+in+Bamboo

So I used Stuart Lange's XSL above as a starting point, and ended up with this:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:msxsl="urn:schemas-microsoft-com:xslt"
                exclude-result-prefixes="msxsl">

  <xsl:output method="xml"
              indent="yes"/>

  <xsl:template match="TestResult">
      <xsl:call-template name="testSuite" />
  </xsl:template>

  <xsl:template name="testSuite">
    <xsl:for-each select="TestSuite">
      <testsuite>
        <xsl:attribute name="errors">
            <xsl:value-of select="@test_cases_failed + @test_cases_aborted"/>
        </xsl:attribute>

        <xsl:attribute name="tests">
            <xsl:value-of select="@test_cases_passed + @test_cases_failed + @test_cases_skipped + @test_cases_aborted"/>
        </xsl:attribute>

        <xsl:attribute name="skipped">
            <xsl:value-of select="@test_cases_skipped"/>
        </xsl:attribute>

        <xsl:attribute name="failures">
            <xsl:value-of select="@test_cases_failed"/>
        </xsl:attribute>

        <xsl:call-template name="testAttributes" />
        <!--results-->
          <xsl:call-template name="testSuite" />
          <xsl:for-each select="TestCase">
            <testcase>
              <xsl:call-template name="testAttributes" />
              <xsl:call-template name="testCaseElements" />
            </testcase>
          </xsl:for-each>
        <!--/results-->
      </testsuite>
    </xsl:for-each>
  </xsl:template>

  <xsl:template name="testAttributes">
    <xsl:attribute name="name">
      <xsl:value-of select="@name"/>
    </xsl:attribute>
    <xsl:attribute name="success">
      <xsl:choose>
        <xsl:when test="@result = 'passed'">True</xsl:when>
        <xsl:when test="@result != 'passed'">False</xsl:when>
      </xsl:choose>
    </xsl:attribute>
    <xsl:attribute name="executed">True</xsl:attribute>
    <xsl:attribute name="time">0</xsl:attribute>
    <xsl:attribute name="asserts">
      <xsl:value-of select="@assertions_failed + @assertions_passed"/>
    </xsl:attribute>
  </xsl:template>

  <xsl:template name="testCaseElements">
    <xsl:if test="@result != 'passed'">
        <failure type="No type reported" message="No message reported"/>
    </xsl:if>
  </xsl:template>

</xsl:stylesheet>

We use the following command line options when we call our Boost.Test executable:

--report_format=xml
--report_level=detailed

It's not the hottest XSL to leave the presses, but it's enough to see the suites, the tests contained, and which ones (if any) failed from Bamboo.

One additional note, with this method you only want to capture the stderr output. The contents of stdout (at least the way we use Boost.Test) will break things.

I need you to log on your other account and give it an email address (the same one as this account) before I can merge. — , Jan 24 '11 at 19:23
How do you transform the XML? Directly in Bamboo? If yes, how did you do it? — Mike, Mar 07 '13 at 16:20
We did the XSL transform as part of an ant task. But there are a number of ways to do it in whatever build workflow you might have going. — joeymink, Jan 13 '14 at 21:22

score 2 · Answer 3 · answered Aug 03 '11 at 14:08

This xsl works for me for converting Boost.Test's log.xml to JUnit xml readable by Bamboo, complete with messages!: https://issues.jenkins-ci.org/secure/attachment/19613/boosttest-1.0-to-junit-1.0.xsl

It's linked from here: https://issues.jenkins-ci.org/browse/JENKINS-7039

score 1 · Answer 4 · edited Nov 20 '11 at 01:06

We run our tests with --report_format=xml --report_level=detailed --log_level=test_suite --log_format=xml. You need both stderr and stdout, then we replace <TestLog> with <xml><TestLog> and replace </TestResult> with </TestResult><xml>. After that we run it though tidy and finally run that though this xslt.

You must also be careful not to have any xml-style tags in your stdout/err. Things like <foo> can break the conversion.

  <xsl:for-each select="./TestSuite">
    <xsl:variable name="name2" select="@name"/>
    <testsuite>
      <xsl:attribute name="errors">
        <xsl:value-of select="@test_cases_failed" />
      </xsl:attribute>
      <xsl:attribute name="tests">
        <xsl:value-of select="@test_cases_failed + @test_cases_passed + @test_cases_skipped" />
      </xsl:attribute>
      <xsl:attribute name="name">
        <xsl:value-of select="@name" />
      </xsl:attribute>
      <xsl:for-each select="./TestCase">
        <xsl:variable name="name3" select="@name"/>
        <testcase>
          <xsl:attribute name="name">
            <xsl:value-of select="@name" />
          </xsl:attribute>
          <xsl:for-each select="/xml/TestLog/TestSuite[@name=$name1]">
            <xsl:for-each select="./TestSuite[@name=$name2]">
              <xsl:for-each select="./TestCase[@name=$name3]">
                <xsl:for-each select="./TestingTime">
                  <xsl:attribute name="time">
                    <xsl:value-of select="./text() div 100000"/>
                  </xsl:attribute>
                </xsl:for-each>
                <xsl:for-each select="./Error">
                  <failure>
                    <xsl:attribute name="type">AssertionFailedError</xsl:attribute>
                    <xsl:attribute name="message">
                      <xsl:value-of select="@file"/>:<xsl:value-of select="@line"/>
                    </xsl:attribute>
                    <xsl:copy-of select="./text()"/>
                  </failure>
                </xsl:for-each>
                <xsl:for-each select="./Exception">
                  <failure>
                    <xsl:attribute name="type">AssertionFailedException</xsl:attribute>
                    <xsl:attribute name="message">
                      <xsl:value-of select="@file"/>:<xsl:value-of select="@line"/>
                    </xsl:attribute>
                    <xsl:copy-of select="./text()"/>
                  </failure>
                </xsl:for-each>
                <system-out>
                  <xsl:copy-of select="./text()"/>
                </system-out>
              </xsl:for-each>
            </xsl:for-each>
          </xsl:for-each>
        </testcase>
      </xsl:for-each>
    </testsuite>
  </xsl:for-each>
</testsuite>

score 0 · Answer 5 · answered Feb 15 '12 at 18:24

Interestingly enough none of the above worked for me, maybe because of newer CruiseControl (2.8.4), but whatever. CruiseControl ignores all the 'errors' and 'failures' attributes that above transforms create, and does it's own queries on the child elements. Now this shows actually how many tests succeeded and which ones failed.

It would be nice to fold in more specific failure info from the boost test log file as well (but maybe somebody else can pick up from here).

Key point: specify following flags to Boost.Test to split out stdout/stderr. Here is a snapshot from an ant build script:

<exec executable="cmd " dir="bin/x64/Release">
<arg line="/k @{file} --build_info --report_format=xml --report_level=detailed --log_level=all --log_format=xml 1&gt; @{file}.log.xml 2&gt; @{file}.result.xml"/>
</exec>

Then transform it:

  <xslt in="@{file}.result.xml" out="@{file}.ccresult.xml" style="transform.xslt" />

The actual transform.xslt:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:msxsl="urn:schemas-microsoft-com:xslt"
                exclude-result-prefixes="msxsl"
>
  <xsl:output method="xml" indent="yes"/>

  <xsl:template match="/TestResult/TestSuite">
    <testsuite>
      <xsl:attribute name="name">
        <xsl:value-of select="@name"/>
      </xsl:attribute>

      <xsl:for-each select="TestCase">
        <testcase>
          <xsl:attribute name="name">
            <xsl:value-of select="@name"/>
          </xsl:attribute>
          <xsl:if test="@result!='passed'">
            <failure>See log file.</failure>
          </xsl:if>
        </testcase>
      </xsl:for-each>
    </testsuite>
  </xsl:template>
</xsl:stylesheet>

Anyone have an XSL to convert Boost.Test XML logs to a presentable format?

5 Answers5