5

My question: Is there a way to force Maven Surefire (or JUnit?) to indicate the name of the failing test class when this class has been executed from a Test Suite?

Now, the long story.

Imagine you have a JUnit test suite that launches two JUnit tests.

Examples of test classes:

public class TestOne {

    @Test
    public void foo() {
        assertTrue(false);
    }

    @Test
    public void bar() {
        assertTrue(false);
    }

}

@UnitTest
public class TestTwo {

    @Test
    public void foo() {
        assertFalse(true);
    }

}

and the test suite:

@RunWith(Suite.class)
@SuiteClasses(value = { TestOne.class, TestTwo.class })
public class MySuite {

}

Now, if I run the test suite (mvn test -Dtest=MySuite), I will have the following errors:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running caf.opm.preclosing.junit.MySuite
Tests run: 3, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec <<< FAILURE!

Results :

Failed tests:
  foo(my.project.junit.TestOne)
  bar(my.project.junit.TestOne)
  foo(my.project.junit.TestTwo)

Tests run: 3, Failures: 3, Errors: 0, Skipped: 0

So reading these lines, I can easily detect which classes and tests are failing.

However, in target/surefire-reports/ directory, I have only one file, MySuite.txt where the three tests are failing. One of the consequences of such situation is that my Hudson build will show me the following information:

All Failed Tests
Test Name       Duration    Age   
>>> my.project.junit.MySuite.foo    0.0 1
>>> my.project.junit.MySuite.bar    0.0 1
>>> my.project.junit.MySuite.foo    0.0 1

As you can see, it becomes harder to identify the failing tests, and for example, as TestOne.foo() and TestTwo.foo() failed, I saw two MySuite.foo errors in my JUnit report. Of course I can retrieve the name of the test class by reading the logs of the failed test, but it's not really a solution...

So my question is to know if I can force Surefire (or JUnit?) to indicate the name of test class.

Thanks.

skaffman
  • 398,947
  • 96
  • 818
  • 769
Romain Linsolas
  • 79,475
  • 49
  • 202
  • 273
  • Do you need to use a custom `TestSuite` - can you have Hudson/Maven simply execute *all* of the tests? The default behavior in the test phase would execute all tests, and report on them as you would expect. – matt b May 17 '11 at 16:11
  • Which version of the surefire plugin do you use? Based on my experience there should be supplemental files in the surefire-reports folder... – khmarbaise May 17 '11 at 16:39
  • Why are you using a suite? Why not let surefire do the work... – khmarbaise May 17 '11 at 16:39
  • @matt b and @khmarbaise If you really want to understand why I need to run a `suite`, read the whole story here: http://stackoverflow.com/questions/4970196/running-all-tests-from-a-category-using-maven – Romain Linsolas May 17 '11 at 20:17
  • I just did a try with Surefire 2.8, but with no luck... – Romain Linsolas May 17 '11 at 20:42

1 Answers1

0

I get a single file per test class, so it took me a moment to figure this one out: You're using a test suite.

When using test suites, all the test results end up in a single file with the name of the suite.

There is no option to change this behavior. Try to file a feature request.

Aaron Digulla
  • 321,842
  • 108
  • 597
  • 820