Junit xml report don't report each test case when 'Karate' parallel runner is used

Question

I am running 5 API test scenarios using karate. When I run the test in non-parallel mode using @RunWith(Karate.class) then in xml generated by surefire-reports, all scenarios are reported individually as .

<testcase classname="[healthCheck]" name="[1:3] Check health check API returns status code 200" time="2.846"/>
  <testcase classname="[healthCheckHeader]" name="[1:6] Check health check API returns status code 200" time="0.285"/>
  <testcase classname="[userLogin]" name="[1:3] Check User Login API returns status code 200" time="0.108"/>
  <testcase classname="[requestChaining]" name="[1:7] chain request demo" time="0.521"/>
  <testcase classname="[viewRequests]" name="[1:10] Check View Requests API returns status code 200" time="0.278"/>

However, when I use karate parallel runner, then each scenario is not reported individually.

<testcase classname="demoTest.AutomationSuiteParallelCucRunner" name="testParallel" time="10.917"/>

I want to have similar report for parallel runner as generated when tests executed in non-parallel mode.

Here is the code for running tests in non-parallel mode:

@RunWith(Karate.class)
public class AutomationSuiteTest {
}

Here is the code for running tests in parallel mode:

@CucumberOptions(tags = {"~@ignore"})

public class AutomationSuiteParallelCucRunner {

    @Test
    public void testParallel() {
        String karateOutputPath = "target/surefire-reports";
        KarateStats stats = CucumberRunner.parallel(getClass(), 3, karateOutputPath);
        assertTrue("SCENARIO FAILURES!!", stats.getFailCount() == 0);
    }

}

The JUnit XML you are looking at is the wrong one, please read the docs carefully: https://github.com/intuit/karate#test-reports Look inside `target/surefire-reports`. Also see this: https://stackoverflow.com/a/57379073/143475 If you still have a problem, follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue — Peter Thomas, Aug 15 '19 at 02:44
Hi Peter, Thanks for your reply. I am looking inside target/surefire-reports and there is xml report for my each scenario like demoTest.userLogin.xml' and there is one xml TEST-demoTest.AutomationSuiteParallelCucRunner.xml'. This file 'AutomationSuiteParallelCucRunner' has different content when I run a test in parallel mode and non-parallel mode. — UT_1, Aug 15 '19 at 05:10

score 1 · Answer 1 · answered Jun 14 '22 at 06:48

Having struggled with this myself, the solution was surprisingly simple, though not really covered in the karate docs (as of this time).

The point is: when using the parallel runner in this manner, there is from JUnit's perspective just one unit test - the single method annotated with @Test. So this single test is all that's listed in target/surefire-reports. Karate doesn't seem to hook into the JUnit execution or so, so that's it.

But - and that was the missing piece for me - the karate runner itself can generate JUnit XML files. You just have to call .outputJunitXml(true) when creating it. This causes it to generate JUnit XML files in the karate report dir along with the HTML report and (optionally) cucumber JSON files:

import com.intuit.karate.Results;
import com.intuit.karate.Runner;

public class KarateTestRunner {
  @Test
  public void runParallel() {
    final Results results = Runner.path("classpath:api/karate")
        .outputCucumberJson(true)
        .outputJunitXml(true)     # <----
        .parallel(8);

    assertEquals(0, results.getFailCount());
  }
}

Junit xml report don't report each test case when 'Karate' parallel runner is used

1 Answers1