Running a JUnit4 Test Suite in Maven using maven-failsafe-plugin

Question

I've got a JUnit 4 test suite that contains a number of test classes in the order they need to be run (our Integration tests need to be run in a certain order).

If I use the maven-failsafe-plugin without any configuration it will run the test but not in the correct order. However, If I set the plugin to run the test suite no tests are run.

Is it possible to run a test suite using the failsafe plugin? if so, where have I gone wrong!!

The code is below:

@RunWith(Suite.class)
@SuiteClasses({ 
    TestCase1.class,
    TestCase2.class, 
       ...
    TestCaseN.class,
})
public class IntegrationSuite {
    //Do Nothing.
}

and from pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.9</version>
    <configuration>
        <includes>
            <include>IntegrationSuite.java</include>
        </includes>
    </configuration>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
            </goals>
    </execution>
    <execution>
        <id>verify</id>
        <goals>
            <goal>verify</goal>
        </goals>
    </execution>
</executions>
</plugin>

Thanks :)

score 11 · Accepted Answer · answered Oct 17 '11 at 12:23

11

Failsafe plugin supports runOrder (click) parameter since version 2.7 (quite recent). There aren't many options, you cannot specify the order explicitly, but you can set it to "alphabetical" and rename your test classes to reflect the run order.

May I also say on the occasion that the fact that test depend on each other is (test) code smell; it's not good, as it is a short path to developing an unmaintainable set of tests and abandoning it finally when its complexity skyrockets above human comprehension. Plus it may fail to expose bugs, as it is a result of one chosen execution path.

BTW, I prefer to include tests like this, with a double asterisk:

<includes>
     <include>**/IntegrationSuite.java</include>
</includes>

answered Oct 17 '11 at 12:23

MaDa

10,511
9
46
84

Thanks for your answer! The tests themselves don't rely on each other to run, however their run order has been determined by a document written detailing the tests... not my decision, but I've still got to implement it!! – Louis Fellows Oct 17 '11 at 13:18
@HeavyMetalKid Now that I think of it, based on how Surefire/Failsafe find test cases, suites are probably not supported. I cannot give a hard proof, though. – MaDa Oct 17 '11 at 13:59
From what I've read whilst looking for an answer to this I think you might be right. I'm gonna try your runOrder + alphabetical naming idea instead. Thanks – Louis Fellows Oct 17 '11 at 14:13
FYI: runOrder-failedfirst is only supported from failsafe version 2.11 – raksja Mar 29 '12 at 22:18

score 0 · Answer 2 · answered Dec 05 '12 at 06:36

maven-surefire-plugin can also be used as below code:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<includes>
<include>**/IntegrationSuite.java</include>
</includes>
</configuration>
</plugin>

Running a JUnit4 Test Suite in Maven using maven-failsafe-plugin

2 Answers2

Linked