Making Maven run all tests, even when some fail

Question

I have a project with several modules. When all tests pass, Maven test runs them all.

When tests fail in the first module, maven will not continue to the next project. I have testFailureIgnore set to true in Surefire settings, but it doesn't help.

How do I make maven run all tests regardless of earlier failures?

testFailureIgnore should work, maybe do a `mvn help:effective-pom` to make sure it's in there in the right place... — rogerdpack, Mar 24 '21 at 21:28

score 455 · Answer 1 · edited Apr 19 '21 at 19:44

455

From the Maven Embedder documentation:

-fae,--fail-at-end Only fail the build afterwards; allow all non-impacted builds to continue

-fn,--fail-never NEVER fail the build, regardless of project result

So if you are testing one module than you are safe using -fae.

Otherwise, if you have multiple modules, and if you want all of them tested (even the ones that depend on the failing tests module), you should run mvn clean install -fn.
-fae will continue with the module that has a failing test (will run all other tests), but all modules that depend on it will be skipped.

edited Apr 19 '21 at 19:44

rogerdpack

62,887
36
269
388

answered Sep 27 '12 at 08:02

despot

7,167
9
44
63

1

additionally you could add -e for the cmd to provide some error information. – despot Sep 27 '12 at 08:03
30

I just tested the option `--fail-never`. Maven will not fail the build even if there are compile errors. If I use this option on Jenkins, the build looks successful even if it has lots of compile errors. I prefer `-Dmaven.test.failure.ignore=true` in this case and let Jenkins analyze the surefire reports. – fuemf5 Jun 25 '18 at 13:07
@wlnirvana I've edited the post now with this link (so feel free to delete your comment...) – Steve Chambers Apr 24 '20 at 10:10
2

Can you configure this in the pom file? – Tigerware Jul 09 '20 at 15:40
@BluE according to [the list of tags](https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html) it doesn't seem so. Maybe there's a way to specify `-fae` directly in `pom.xml?` – nmd Feb 20 '21 at 15:22
Testing "one" module I think doesn't fit...With one module test the build will be interrupted when a failure (unit test, compile) occurs with or without either setting. Unless you're using `-Dmaven.test.failure.ignore=true`. However it does effect modules as described (skips dependent ones unless you use `--fail-never`). `--fail-never` happens to also return a "success" process exit code FWIW :) – rogerdpack Mar 24 '21 at 21:21
Sadly neither of these is enough if there are "unresolvable dependencies" or pom syntax error type failures. Oh well... – rogerdpack Mar 24 '21 at 21:32

score 117 · Answer 2 · edited Mar 24 '21 at 20:42

117

Either configure Surefire with <testFailureIgnore>true</testFailureIgnore>.

Or on the command line:

mvn install -Dmaven.test.failure.ignore=true

edited Mar 24 '21 at 20:42

rogerdpack

62,887
36
269
388

answered Nov 14 '10 at 07:36

Pascal Thivent

562,542
136
1,062
1,124

24

IIRC this has the effect of the whole build not failing, rather than running all tests and failing in the end. – Ondra Žižka Nov 03 '11 at 18:23
2

This is the only option that worked for me ... The "-fae" option did nothing on test failures – Yoi Mar 17 '17 at 15:58
Yes if you only have unit test failures this will make the whole build "look like" it passed with success. `--fail-at-end or -fae` should make it continue on and just fail the build at the end, though it also skips dependent modules see https://stackoverflow.com/a/12616700/32453 – rogerdpack Mar 24 '21 at 21:26
The great thing about this option is that you can specify it under "Global Execution Options" for Maven in NetBeans IDE, which is great when you can't go around modifying every module's POM (I'm working on a project with 198 modules). – Nathan Crause Feb 17 '22 at 17:54

score 116 · Accepted Answer · edited Apr 13 '21 at 17:02

116

I just found the -fae parameter, which causes Maven to run all tests and not stop on failure.

edited Apr 13 '21 at 17:02

Sylhare

5,907
8
64
80

answered Nov 13 '10 at 21:18

ripper234

222,824
274
634
905

4

Sadly, this parameter doesn't work when I pass it to TeamCity. – ripper234 Nov 13 '10 at 21:25
10

In a multi-module project, modules that depend on module that has failed tests will be skipped. Use -fn instead. – hipokito Mar 09 '18 at 07:17
2

@ripper234 did you found a way to make it work on TeamCity? – Jeredriq Demas Jun 25 '18 at 07:55
Can you configure this in the pom file? – Tigerware Jul 09 '20 at 15:40
This only works for modules that don't have dependencies on other modules within the same build tree, see https://stackoverflow.com/a/12616700/32453 – rogerdpack Mar 24 '21 at 21:29

score 48 · Answer 4 · answered Nov 24 '10 at 07:51

48

Try to add the following configuration for surefire plugin in your pom.xml of root project:

<project>
  [...]
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
        </configuration>
      </plugin>
    </plugins>
  </build>
  [...]
</project>

answered Nov 24 '10 at 07:51

nybon

8,894
9
59
67

As explained in other responses, this will execute all tests but also mark the build as success even if there are failures – laffuste Jul 12 '21 at 07:17

score 22 · Answer 5 · answered Apr 17 '14 at 15:22

22

A quick answer:

mvn -fn test

Works with nested project builds.

answered Apr 17 '14 at 15:22

rustyx

80,671
25
200
267

Is this better than `mvn install -Dmaven.test.failure.ignore=true`? – Paul Feb 20 '23 at 20:50

Making Maven run all tests, even when some fail

5 Answers5

Linked

Related