Surefire rerun failing tests not working

Question

I want to rerun a test that I know will fail cause I am trying to test the Surefire parameter for re-running the failing tests. I tried running Maven with these two commands neither of them works as expected

-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails test

and

-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails surefire:test

Here is part of pom.xml

<dependency>
    <groupId>org.apache.maven.surefire</groupId>
    <artifactId>surefire-api</artifactId>
    <version>2.19.1</version>
</dependency>
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>2.53.1</version>
</dependency>

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>

I was expecting that Surefire would restart the test after failure but Maven just throws this error, which I know how to solve but I want the test to be rerun.

Results :

Tests in error: 
  testA(selenium.services.TestThatWillFail): Element is not currently visible and so may not be interacted with(..)

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 55.060 s
[INFO] Finished at: 2016-11-24T12:58:02+01:00
[INFO] Final Memory: 18M/173M

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project eskn_selenium: There are test failures.

score 15 · Answer 1 · edited Oct 27 '17 at 12:47

15

Just to add to Wim Rutgeerts's answer - rerunFailingTestsCount must be in the configuration section, not in properties, like this:

<configuration>
    <rerunFailingTestsCount>2</rerunFailingTestsCount>
</configuration>

In my case with maven-surefire-plugin 2.19.1 it worked this way. When it was in properties it did not work.

edited Oct 27 '17 at 12:47

Adil B

14,635
11
60
78

answered Oct 27 '17 at 12:20

SerhiiBond

247
3
8

score 5 · Answer 2 · answered Apr 07 '17 at 10:15

5

Instead of using the command line property -Dsurefire.rerunFailingTestsCount=2, you can also define it in the pom in the properties section

 <properties>
    <surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
 </properties>

answered Apr 07 '17 at 10:15

Wim Rutgeerts

109
2
5

Lucian Radu · Answer 3 · 2022-11-10T12:38:34.843

5

Update for JUnit 5: Maven Surefire version 3.0.0-M4 or later now lets you use rerunFailingTestsCount system property when executing JUnit 5 tests.

Make sure to pass the below property when running your mvn clean stage:

-Dsurefire.rerunFailingTestsCount=3

edited Nov 10 '22 at 12:38

answered Aug 04 '20 at 20:26

Lucian Radu

302
1
4
17

1

You don't have to pass the property, you can also put it in `` as documented in other answers. – rjmunro Jun 09 '21 at 16:43

score 4 · Accepted Answer · answered Nov 24 '16 at 12:19

4

Although that is missing from the documentation, the parameter rerunFailingTestsCount was introduced in version 2.18 of the Maven Surefire Plugin, as mentioned in SUREFIRE-1087. Since you're using the default version of 2.12.4 (that comes from the Super POM), that option is not available.

Therefore, the fix is simply to update the Surefire version to a version that is at least 2.18; for example, the latest, which is currently 2.19.1:

<pluginManagement>
  <plugins>
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.19.1</version>
    </plugin>
  </plugins>
</pluginManagement>

Note that this parameter only works with JUnit 4+ (which is your case, since you have JUnit 4.12).

answered Nov 24 '16 at 12:19

Tunaki

132,869
46
340
423

oooooh now I see ! I thought that adding it as dependency will do the work but it has to be added as plugin. I was about to ask if you did not notice that it is already defined in pom.xml haha. Thank you! – hocikto Nov 25 '16 at 07:39
4

Update for **JUnit 5**: Maven Surefire `3.0.0-M5` now lets you use `rerunFailingTestsCount` system property when executing tests. Make sure to pass the below property when running your `mvn clean` stage: ```-Dsurefire.rerunFailingTestsCount=3``` – Lucian Radu Aug 04 '20 at 20:30
See the Feature Matrix for Surefire/Failsafe https://maven.apache.org/surefire/maven-surefire-plugin/featurematrix.html – tibor17 Oct 05 '20 at 17:49

Surefire rerun failing tests not working

4 Answers4

Linked