How to get CMocka report in JUnit format?

Question

I am able to use cmocka and getting default results on the screen. I want to get the results for the unit test in JUnit format.

CMocka supports JUnit format by using the env variable CMOCKA_MESSAGE_OUTPUT or using API cmocka_set_message_output(CM_OUTPUT_XML);

But still no xml file gets generated. Can anyone help out in obtaining results in JUnit format?

Answer 1

The most credible source is actually the cmocka.c source file that contains entire implementation of the framework. This file is not too large, so I will cite something from the source file version 1.0.1.

There are two conditions to generate XML output by cmocka and the third condition is needed to store output in file.

1. XML output can be generated only if tests are called by cmocka_run_group_tests()

The customizable output format can be obtained only from the test runner cmocka_run_group_tests() or from its full variation cmocka_run_group_tests_name().

There is no other route that can lead to XML output. If a singe test is started by run_test() the output cannot be XML.

The summary format

[  PASSED  ] 0 test(s).
[  FAILED  ] 1 test(s), listed below:

can be generated in one of the following possible cases:

• the test is started by one of the deprecated test runners: run_tests(), _run_tests() or run_group_tests(), _run_group_tests(); in that case it is even possible to see compilation warning about usage of a deprecated function;
• the test is started by cmocka_run_group_tests() and the output format is CM_OUTPUT_STDOUT.

2. cmocka message output should be set to CM_OUTPUT_XML

The default output format can be set by calling cmocka_set_message_output(CM_OUTPUT_XML) before running tests. However, even if such default is set in the test source it can be overwritten by the environment variable CMOCKA_MESSAGE_OUTPUT. That variable has higher priority than the default set by cmocka_set_message_output().

The value of CMOCKA_MESSAGE_OUTPUT is case insensitive. The variable is taken into account if it is equal to one of the following values: stdout, subunit, tab or xml.

So, if the environment variable has value stdout the function mocka_set_message_output() has no effect.

That variable can be used to force different output formats of already compiled binary:

CMOCKA_MESSAGE_OUTPUT=stdout ./nulltest
CMOCKA_MESSAGE_OUTPUT=subunit ./nulltest
CMOCKA_MESSAGE_OUTPUT=tap ./nulltest
CMOCKA_MESSAGE_OUTPUT=xml ./nulltest

Thus, if the test is started by cmocka_run_group_tests() but the output is not affected by mocka_set_message_output() it means that there is set variable CMOCKA_MESSAGE_OUTPUT=stdout in the shell.

3. It should be possible to create a new file by cmocka to write its XML output directly to that file

If both previous conditions are satisfied, it is possible to ask cmocka to write its XML output directly to a file. If the environment variable CMOCKA_XML_FILE is set, then cmocka will try to write XML to file with name of that variable value.

Usage example:

CMOCKA_XML_FILE='./out.xml' CMOCKA_MESSAGE_OUTPUT=xml ./nulltest

The file is written if:

• file with such name does not exist;
• such file can be created.

Thus if there are more than one test runners in one compiled binary test application, only the first runner can write its output to that file.

The output is written to the shell even if CMOCKA_XML_FILE is set, but the file already exists or it cannot be created.

Of course, it is possible just to redirect shell output to a file overwriting existent file or appending to existent file if such file exists.

The example bellow can be used to check different options for output. It can be built by the command

gcc -g nulltest.c -o nulltest -Ipath_to_cmocka_headers -Lpath_to_cmocka_library_binary -lcmocka

nulltest.c

#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>
#include <cmocka.h>

/* A test case that fails. */
static void null_test_failed(void **state) {
    (void) state; /* unused */
    assert_int_equal(0, 1);
}

int main(void) {
    const struct CMUnitTest tests[] = {
        cmocka_unit_test(null_test_failed),
    };
    const struct UnitTest tests_deprecated[] = {
        unit_test(null_test_failed),
    };

    cmocka_set_message_output(CM_OUTPUT_XML);

    /* group test functions that use customizable output format */
    cmocka_run_group_tests(tests, NULL, NULL);
    cmocka_run_group_tests_name("custom group name", tests, NULL, NULL);

    /* run single test with standard output */
    run_test(null_test_failed);

    /* DEPRECATED TEST RUNNER functions that can give only standard output */
    run_tests(tests_deprecated);
    _run_tests(tests_deprecated, 1);
    run_group_tests(tests_deprecated);
    _run_group_tests(tests_deprecated, 1);

    return 0;
}

Answer 2

The XML is printed to stdout, you need to redirect it to a file ...