How to run unit tests in excludedGroups in Maven

Question

I have a JUnit 4.12 SlowTests test suite that I want to exclude from execution unless specifically requested on the Maven command line.

I have added the following to my pom file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19</version>
    <configuration>
        <excludedGroups>com.Example.SlowTests</excludedGroups>
        <includes>
            <include>**/*TestSuite.class</include>
        </includes>
        <excludes>
            <exclude></exclude>
        </excludes>
    </configuration>
</plugin>

I have defined the category as SlowTests and applied it to the MySlowTests class.

I have annotated the test suite as follows:

@RunWith(Categories.class)
@IncludeCategory(SlowTests.class)
@SuiteClasses({ MySlowTests.class })
public class MySlowTestSuite

When I run mvn package all the unit tests except MySlowTests are executed.

However, looking at various answers such as https://stackoverflow.com/a/25639372/820657 and https://stackoverflow.com/a/21830866/820657 I expected the following command:

mvn package -Dgroups=com.Example.MySlowTests

to run the excluded MySlowTests tests but they don't run. In fact no tests run.

What am I doing wrong?

Answer 1

The Maven Surefire plugin has some issues w.r.t categories in versions < 2.13 (IIRC) but as long as you are using Surefire >= 2.13 the following will run any classes annotated with @Category(com.yourcompany.SlowTests.class):

<plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.13</version>
      <configuration>
        <groups>com.yourcompany.SlowTests</groups>
      </configuration>
</plugin>

This approach is often used with profiles, the following configuration ...

<profiles>
    <profile>
        <id>slow</id>
        <properties>
            <testCategories>com.yourcompany.SlowTests</testCategories>
        </properties>
    </profile>
    <profile>
        <id>fast</id>
        <properties>
            <testCategories>com.yourcompany.FastTests</testCategories>
        </properties>
    </profile>
</profiles>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.13</version>
            <configuration>
                <groups>${testCategories}</groups>
            </configuration>
        </plugin>
    </plugins>
</build>

... can be used to run:

• mvn install -P slow: runs the slow tests only
• mvn install -P fast: runs the fast tests only
• mvn install -P fast,slow: runs the fast and slow tests

Update 1: for this question: "Is there a way to use this approach so I can run all fast tests by default?"

You can define two properties:

<properties>
    <includedCategories></includedCategories>
    <excludedCategories>com.yourcompany.SlowTests</excludedCategories>
</properties>

Then update your surefire plugin definition like so:

    <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.13</version>
        <configuration>
            <groups>${includedCategories}</groups>
            <excludedGroups>${excludedCategories}</excludedGroups>
        </configuration>
    </plugin>

And, finally, add this profile:

    <profile>
        <id>slow</id>
        <properties>
            <includedCategories>com.yourcompany.SlowTests</includedCategories>
            <excludedCategories></excludedCategories>
        </properties>
    </profile>

This just toggles the includedCategories and excludedCategories properties. By default, you include everything except those tests annotated with com.yourcompany.SlowTests (i.e. everything except your 'slow' tests). When you run with -P slow you exclude everything except those tests annotated with com.yourcompany.SlowTests (i.e. everything except your 'slow' tests).

Note: what I said in my original answer about Surefire versions < 2.13 misbehaving with Categories still stands so to make this work you need to be using a version of the Maven Surefire plugin >= 2.13.