Prevent unit tests but allow integration tests in Maven

Question

I've a Maven build in which I use the SureFire plugin to run some unit tests, and the FailSafe plugin to run some integration tests. I would like a way to run just the FailSafe plugin's tests.

It's not a good solution for me to add different profiles or anything in the pom, because it's a multimodule build and I don't want to have to edit every module's pom.

There are skip.tests and maven.test.skip and skipTests which stop all tests, and skipITs, which stops only the failsafe plugin.

So, is there a command-line flag for Maven like skipITs, but instead with the functionality of "onlyITs"?

What is the exact reason to just run the integration test? Usually Unit tests are very fast so not a real problem? — khmarbaise, Jul 07 '11 at 14:41
@khmarbaise in theory, yes. But in most projects I have worked in, the "unit tests" where actually integration tests with an in-memory db (if you were lucky) — Sean Patrick Floyd, Jul 07 '11 at 14:43
@khmarbaise Lots of unit tests. They take a couple of minutes to run and we don't need them to run in this circumstance. Specifically, we run unit tests before building the artifact (of course), but we want to run the ITs in multiple environments. No point re-running the unit tests at this point. — Matthew Gilliard, Jul 07 '11 at 14:45
If you have to run your integration test into different environments i would suggest to put the integration test into a separate Maven Module instead of combining that with Unit Tests. The problem is that Maven will run the unit tests usually in it's life cycle...? Other question: How many Unit tests do you have? (May you have to make the unit tests faster?) — khmarbaise, Jul 07 '11 at 16:13
Hi @khmarbaise, in my set up `skipTests` skips only the surefire tests, not the failsafe test! Maybe it's a new feature ? — danidemi, May 15 '17 at 15:36
FYI : skipTests is now deprecated in Failsafe Plugin 3.0.0-M3 ([SUREFIRE-1611](https://issues.apache.org/jira/browse/SUREFIRE-1611)) — Guillaume Husta, Feb 01 '19 at 10:49
Have Failsafe Plugin 3.0.0-M5 and it still skip integration tests with `skipTests`. According to the comment in source code `skipTests` will be removed in Failsafe 3.0.0 — Ilya Serbis, Sep 11 '20 at 18:10

bacar · Accepted Answer · 2013-07-30T12:22:45.613

I found the simplest way to skip only surefire tests is to configure surefire (but not failsafe) as follows:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.14</version>
    <configuration>
        <!-- skips surefire tests without skipping failsafe tests.
                 Property value seems to magically default to false -->
        <skipTests>${skip.surefire.tests}</skipTests>
    </configuration>
</plugin>

This allows you to run mvn verify -Dskip.surefire.tests and only surefire, not failsafe, tests will be skipped; it will also run all other necessary phases including pre-integration and post-integration, and will also run the verify goal which is required to actually fail your maven build if your integration tests fail.

Note that this redefines the property used to specify that tests should be skipped, so if you supply the canonical -DskipTests=true, surefire will ignore it but failsafe will respect it, which may be unexpected, especially if you have existing builds/users specifying that flag already. A simple workaround seems to be to default skip.surefire.tests to the value of skipTests in your <properties> section of the pom:

<properties>
    <skip.surefire.tests>${skipTests}</skip.surefire.tests>
</properties>

If you need to, you could provide an analagous parameter called skip.failsafe.tests for failsafe, however I haven't found it necessary - because unit tests usually run in an earlier phase, and if I want to run unit tests but not integration tests, I would run the test phase instead of the verify phase. Your experiences may vary!

These skip.(surefire|failsafe).tests properties should probably be integrated into surefire/failsafe code itself, but I'm not sure how much it would violate the "they're exactly the same plugin except for 1 tiny difference" ethos.

Using this solution I've been able to set up my framework so -DskipUnitTests skips over the surefire plugin, -DskipIntegrationTests skips over the failsafe plugin, and DskipTests skips over both. Exactly what was needed! — Alex Jansen, Sep 13 '14 at 01:54
my IDE is complaining about "cannot resolve symbol 'skipTests'" the solution was to add a line `false` still works with any combination of -DskipTests or -Dskip.surefire.tests as command line args seem to overwrite properties https://stackoverflow.com/questions/13708738/how-to-get-a-command-line-property-to-overwrite-a-maven-property you may want to add that to your solution — globalworming, Sep 13 '17 at 07:20
`${skip.surefire.tests}` is not working with `maven-surefire-plugin` version `3.0.0-M3`. All the surefire tests are still running. Anyone else found this? Sean Patrick Floyd's below solution is however working. — John, Jul 14 '19 at 16:38
About "however I haven't found it necessary - I would run the test phase instead of the verify phase.", this would disable a test coverate analysis run by Jacoco in the "verify" phase, and u may want to see your Unit test coverage without integration tests. — Tristan, May 11 '23 at 08:03

Sean Patrick Floyd · Answer 2 · 2011-07-07T14:53:57.323

131

A workaround would be to call:

mvn clean test-compile failsafe:integration-test

Admittedly, this is ugly, but it may solve your problem.

Or (another hack):

mvn clean integration-test -Dtest=SomePatternThatDoesntMatchAnything -DfailIfNoTests=false

Reference:

edited Jul 07 '11 at 14:53

answered Jul 07 '11 at 14:47

Sean Patrick Floyd

292,901
67
465
588

7

Is this a good idea? **Won't this result in your build succeeding even if your integration tests fail?** That's the whole essence of failsafe, [if you don't *also* run the 'verify' goal](http://maven.apache.org/surefire/maven-failsafe-plugin/). Quote: "The Failsafe Plugin will not fail the build during the integration-test phase". You need run the verify goal to actually tell you whether the integration tests succeeded or not! – bacar Jul 29 '13 at 17:58
2

@bacar is right, but just use `verify` instead of `integration-test` in the 2nd solution. – Matthew Gilliard Sep 18 '13 at 08:05
@MatthewGilliard still a bit hacky, if you have no good reason to be setting `-DfailIfNoTests=false` anyway (failsafe will respect this flag too)... but I'm groping for flaws now :-) – bacar Sep 26 '13 at 15:40
13

If you add `failsafe:verify` to the end of the first hack (`mvn clean test-compile failsafe:integration-test failsafe:verify`) it will fail the build if one of the integration tests fails. – Shadow Man Jun 05 '14 at 20:21
1

The first solution also works with running a single test: `mvn -Dit.test= failsafe:integration-test` – Kat Aug 01 '14 at 20:41
One reason to skip (huge piles of) unit tests and ignore verify is that one is debugging an integration test or a bug that it has uncovered. – Mark Wood Jun 07 '16 at 20:36
`mvn clean test-compile failsafe:integration-test` is working with version `3.0.0-M3`. – John Jul 14 '19 at 16:41
A reasonable choice for `SomePatternThatDoesntMatchAnything` is this: `!*`, matching nothing. – gebi Jan 21 '20 at 20:25

Martijn Burger · Answer 3 · 2018-09-22T12:21:59.277

I am using the code from Antonio Goncalves Blog , which works perfect.

You can use the following properties:

-DskipUTs=true for skipping surefire tests.

-DskipITs=true for skipping failsafe tests.

-DskipTests=true for skipping all tests.

The pom.xml is as follows:

<properties>
    <skipTests>false</skipTests>
    <skipITs>${skipTests}</skipITs>
    <skipUTs>${skipTests}</skipUTs>
</properties>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <skipTests>${skipUTs}</skipTests>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.19.1</version>
            <executions>
                <execution>
                    <id>run-integration-tests</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <skipTests>${skipTests}</skipTests>
                <skipITs>${skipITs}</skipITs>
            </configuration>
        </plugin>
    </plugins>
</build>

thanks, great approach! it also useful to override default execution of surefire (default-test) see: http://stackoverflow.com/questions/11935181/what-does-the-default-test-stand-for-in-the-maven-surefire-plugin — pls, Sep 25 '16 at 10:48
Really a nice solution. However, the failsafe configuration is redundant as `skipITs` is the default. — timomeinen, Apr 19 '18 at 19:42
SkipITs is a custom option in this config. You right in the default implementation, but that was not the OPs question. — Martijn Burger, Apr 19 '18 at 19:47

score 31 · Answer 4 · answered Dec 01 '13 at 15:32

31

Hope this helps!

Try to run test only with FailSafe (plugin for integration-test - it will allow you to run only integration tests with this kind naming, by default: */IT.java, **/IT.java, */*ITCase.java; , but you can easily change that from pom file)

mvn failsafe:integration-test

And when you want to use only SureFire (plugin for unit-testing)

mvn surefire:test

or one test at a time with:

mvn -Dtest=MyUnitlTest

answered Dec 01 '13 at 15:32

Kati Holasz

498
5
8

This should have more votes! Simple and works! – Pubudu Nov 16 '22 at 10:29
This works but disables other plugins who run only in the "verify" phase like test coverage analysis with Jacoco. – Tristan May 11 '23 at 08:04

score 15 · Answer 5 · answered Feb 28 '14 at 11:19

15

I do like this so every phases are normally executed:

 mvn -Dtest=foo -DfailIfNoTests=false verify

answered Feb 28 '14 at 11:19

Jérôme Herry

187
2
7

2

doesn't require adding anything in the pom as the OP asked and runs all phases as indicated. great answer. – jnichols959 Jul 16 '19 at 23:48
This trick is underrated. Thanks for the tip. – kntx Oct 05 '21 at 21:06

score 2 · Answer 6 · answered Feb 02 '21 at 10:19

2

This skips the UnitTests triggered by SureFire and tests only one test:

mvn failsafe:integration-test -Dit.test=YourTestName

You can also provide a pattern: mvn failsafe:integration-test -Dit.test=*

(Maven 3.6.3)

answered Feb 02 '21 at 10:19

dr0i

2,380
2
19
36

score -2 · Answer 7 · answered Jul 08 '11 at 00:00

-2

Try running your integration or unit tests in a separate profile. Then you can just enable/disable the profile.

answered Jul 08 '11 at 00:00

Steven

3,844
3
32
53

Maybe add parent pom in which you define profile that is running only ITs? all project sub modules could inherit form that pom, so you wouldn't need to change every pom or run modules with special switches (as you can activate profile on property absence). – yoosiba Jul 08 '11 at 09:13

Prevent unit tests but allow integration tests in Maven

7 Answers7

Linked