Maven: Ignore inter-module dependencies while running unit tests

Question

We have a large multi-module Maven project. I've been experimenting with speeding up our unit test builds using the -T option, with some positive results. However, there are some dependency paths in our project like this:

module A <- module B <- module C

where the unit tests for each module take 20-30 minutes. Since the -T option builds modules in their dependent order, this results in a 90 minute total build time. It would really speed up the build if I could compile all the modules first, and then run the tests for A, B, and C in parallel. Eg. Something like this:

$ mvn -T 10 clean install -DskipTests
$ mvn -T 10 --ignore-dependencies test

Question: Does Maven have support for this out of the box?

I've been toying with the idea of writing a tiny script that will parse the output of mvn dependency:tree and invoke "mvn test -pl A", "mvn test -pl B" and so forth in parallel, but obviously if Maven has an out-of-the-box solution that'd be preferable.

We are using Jenkins, so if there's some Jenkins plugin out there or feature of Jenkins that I've missed that supports this, that could help a lot!

Note: Speeding up the unit tests for A, B and C will take a significant amount of work, and there's no guarantee that the tests within an individual module are parallelizable

Answer 1

A possible solution would be as following:

• Create an additional module, say testsuite-module
• Add to the testsuite-module all other modules as dependencies in test scope
• Add to the testsuite-module all the test sources of other modules via the Build Helper Maven Plugin and its add-test-source goal
• Execute the second step of your job only on this module and run tests in parallel

For instance, the POM file of the testsuite-module may look as following:

<project>
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.sample</groupId>
        <artifactId>modules</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <artifactId>testsuite-module</artifactId>

    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <version>1.10</version>
                <executions>
                    <execution>
                        <id>add-test-source</id>
                        <phase>generate-test-sources</phase>
                        <goals>
                            <goal>add-test-source</goal>
                        </goals>
                        <configuration>
                            <sources>
                                <source>../module-a/src/test</source>
                                <source>../module-b/src/test</source>
                            </sources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>com.sample</groupId>
            <artifactId>module-a</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.sample</groupId>
            <artifactId>module-b</artifactId>
            <version>0.0.1-SNAPSHOT</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

Its only scope is to gather all source tests of other modules, having the required modules in test scope/classpath and execute them via for instance:

mvn -pl testsuite-module test -T 10

This would execute all of the tests in a single test execution and as such possibly meet your requirements.

Few considerations on this approach:

• The testsuite-module would be harmless for your project and your could also move it to a CI profile, if required (and recommended), as explained in this SO post
• You may consider to also use the add-test-resource goal of the Build Helper plugin
• You may have conflicts on test names (two test cases with the same name in different modules) or on test resources though, this may be more of an issue but shouldn't be impossible to sort it out
• You may waste time (and be counter-productive) if tests from dependent modules would fail first, but this aspect was already foreseen by your requirement (as assumption, I suppose)