Maven fail-safe not executing tests

Question

I've combed StackOverflow and many other sites, have found many other related posts and have followed all said suggestions, but in the end, failsafe is skipping my tests.

My JUnit test is located here: myModule/src/main/test/java/ClientAccessIT.java

I am skipping surefire because there are no unit tests in this module:

<!-- POM snippet -->
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
  <skip>true</skip>
  </configuration>
</plugin>

And I'm trying to run integration tests with failsafe:

<!-- POM snippet -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <executions>
        <execution>
            <id>run-tests</id>
            <phase>integration-test</phase>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

However, when I run mvn verify I see this:

[INFO] --- maven-failsafe-plugin:2.14.1:integration-test (run-tests) @ rest-services-test ---

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

I spent the last 4 1/2 hours scouring, any help would be appreciated. The only other thing that may be worth mentioning is that I have Cargo setting up and tearing down a Tomcat container. Does anybody see the glaring problem?

Why isn't `maven-failsafe-plugin` running out of the box on the `integration-test` phase? — AlikElzin-kilaka, Jun 25 '21 at 07:04

score 29 · Answer 1 · answered May 09 '17 at 17:02

29

Your tests are not in the default test sources directory src/test/java. See:

https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

myModule/src/main/test/java/ClientAccessIT.java

should be:

myModule/src/test/java/ClientAccessIT.java

You could also update your pom file (if you really wanted tests to live in main) to include:

<build>
    <testSources>
        <testSource>
            <directory>src/main/test</directory>
        </testSource>
    </testSources>
</build>

answered May 09 '17 at 17:02

pro-tester

371
1
4
2

2

This should be the accepted answer. The given example clearly doesn't work because of this. – Thomas Naskali Nov 21 '18 at 11:23
4

Did you mean `` or ``? I cannot find element ` – Nom1fan Dec 19 '19 at 16:11
This has changed. Since failsafe v2.2.: https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testSourceDirectory – Ueffes May 11 '22 at 09:11

neXus · Answer 2 · 2022-02-07T13:26:34.317

22

If you happen to use Spring Boot version 2.4 or higher, and your tests still use JUnit 4, remember to put this dependency:

<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <scope>test</scope>
</dependency>

Version 2.4 of spring-boot-starter-test includes only JUnit 5 and removed the vintage engine that provided a TestEngine for running JUnit 3 and 4 based tests.

Without that vintage engine, all JUnit 4 tests are silently ignored.

edited Feb 07 '22 at 13:26

answered Mar 29 '21 at 09:02

neXus

2,005
3
29
53

Thanks neXus for this answer. It saved lot of time, since it was very difficult to find the cause, as no test where executed for me as all of the test where written in Junit4. – Suvojit Sep 24 '21 at 11:47
1

Or update to junit 5: https://stackoverflow.com/questions/65073944/it-are-no-longer-executed-with-failsafe-plugin-after-migrating-to-spring-boot-2/70991122#70991122 :) – rogerdpack Feb 04 '22 at 18:35
This was my issue. Some how whoever set up our integration tests got them on an old version of jUnit, while the unit tests were on jUnit 5 and worked just fine. So my mvn verify would execute unit test but int test would have 0 test found. – dsmithpl13 Mar 15 '23 at 17:18

score 18 · Answer 3 · answered Jul 31 '15 at 21:11

18

I had a similar problem. If there aren't any test classes compiled to target/test-classes then check your pom file and ensure that the packaging isn't 'pom'.

answered Jul 31 '15 at 21:11

Raymond

406
3
9

What is "packaging isn't pom"? Any hint? – markthegrea Sep 12 '18 at 18:40
Look in your effective pom.xml for the tag "packaging". Look at this link for a bit more about the packaging tag. https://maven.apache.org/pom.html. the key is to ensure that the value of that tag is not "pom". – Raymond Sep 13 '18 at 21:55

score 17 · Answer 4 · answered Aug 01 '16 at 19:49

17

For multi-module projects, the child project needs to have the plugin declared as followed,

 <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
 </plugin>

version inherits from parent pom. Without the above defined, the plugin in parent pom alone will not work.

answered Aug 01 '16 at 19:49

barryku

2,496
25
16

I had to define plugin inside parent's pom (pluginManagement section) and use it on the project's pom (plugins section) – Sergio Gabari Sep 04 '18 at 08:30

Keppil · Accepted Answer · 2013-05-05T20:26:56.073

12

You need to rename your test class.

You can find the names the plugin looks for by default in the documentation, as pointed out by @acdcjunior:

By default, the Failsafe Plugin will automatically include all test classes with the following wildcard patterns:

"**/IT*.java" - includes all of its subdirectories and all java filenames that start with "IT".

"**/*IT.java" - includes all of its subdirectories and all java filenames that end with "IT".

"**/*ITCase.java" - includes all of its subdirectories and all java filenames that end with "ITCase".

edited May 05 '13 at 20:26

answered May 05 '13 at 18:45

Keppil

45,603
8
97
119

4

That does not seem to be true: http://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html - it says the default patterns are `**/IT*.java`, `**/*IT.java`, `**/*ITCase.java`. – acdcjunior May 05 '13 at 20:17
1

@acdcjunior beat me to it; here's a direct link to the relevant documentation page: https://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#includes – kryger May 05 '13 at 20:21
2

"**/*IT.java" - includes all of its subdirectories and all java filenames that end with "IT". Wouldn't ClientAccessIT.java meet the qualifications of this pattern? – dingalla May 05 '13 at 23:01
32

The example "ClientAccessIT.java" clearly matches *IT.java. So this answer is just useless and confusing. – Gustave Feb 15 '17 at 06:50
if you are usin JUnit 5 you can include them by filtering and using tags: https://maven.apache.org/surefire/maven-failsafe-plugin/examples/junit-platform.html# – Sergio Gabari Sep 04 '18 at 08:47
4

This answer is wrong and misleading. The correct answer was given by @pro-tester – t0r0X Feb 27 '20 at 23:20

score 9 · Answer 6 · edited Feb 04 '22 at 18:51

I was having the same issue and tried a few of the suggested options here. I am developing a Spring Boot application using JUnit 5. Unfortunately, none of my integration tests (located in src/test/java and suffixed with *TestIT.java) were getting picked up, regardless of the advertising of the plugin to do so. I tried downgrading to 2.18.1 and 2.19.1 and tried both removing the <executions> and <configuration> sections to attempt to use even the plugin's default functionality, but no luck.

I finally changed the version to the latest (as of the time of writing) to 2.22.0, and viola. Here is the configuration I added to the <build><plugins> section of my pom. And even better, I don't need to add executions that define the integration-test and verify goals that you see most people include; failsafe executes these by default. I've included my surefire plugin configuration as well, since I am using it to execute unit tests during the test phase of the maven lifecycle.

Edit: I also had to ad the integration-test and verify executions to the plugin configuration as well.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.0</version><!--$NO-MVN-MAN-VER$-->
</plugin>

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.22.0</version><!--$NO-MVN-MAN-VER$ -->
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
     </execution>
  </executions>
</plugin>

And just for good measure, to tell surefire and failsafe to use JUnit 5, I am including this in my <dependencies> section:

<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter-engine</artifactId>
  <scope>test</scope>
</dependency>

Eclipse displays a warning (yellow squiggly underline) under the version tag, hence the inclusion of the  error. I'm sure there's a better way to deal with this, but it works for me in this case.

To speed up testing this configuration, and to avoid having to go through all of the earlier lifecycle phases before test, integration-test orverify phases, I used the following commands to quickly validate:

mvn failsafe:integration-test (runs only integration tests)
mvn surefire:test (runs only unit tests)
(I obviously run the entire lifecycle when it counts)

Hope this helps someone. Cheers!

For me nothing was working without this junit-jupiter-engine dependency added in tests module. Thanks for the hint! Although I don't fully understand yet how it actually influenced it and made it work... Can anyone explain? When I started these tests by surfire-plugin (in default test phase), then the dependency wasn't needed and everything worked. — alwi, Jul 17 '19 at 07:53
See the comment from Horatiu. He has the right solution, as by the Maven references: integration-test and verify — tibor17, Jul 02 '20 at 13:36
Interesting. I used to have to downgrade to 2.18.1. Which still works for me. Unless your tests are junit 5 then I have to use M-3.0.x https://stackoverflow.com/a/70991122/32453 — rogerdpack, Feb 04 '22 at 18:56

score 6 · Answer 7 · answered Feb 17 '16 at 12:31

6

I also had a similar problem but needed the packaging element to be "pom". Make sure your test source files are being compiled and added to the .../target/test-classes folder using the maven-compiler-plugin:

       <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>${maven.compiler.version}</version>
            <executions>
                <execution>
                    <phase>test-compile</phase>
                    <goals>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

answered Feb 17 '16 at 12:31

bonapart3

469
7
20

Strangely my packaging was jar but test classes were not compiled. Thanks. – Vitaly Sazanovich Oct 12 '16 at 14:46
This was my issue also, as soon as I included compiler plugin integration tests started working! – Leon Sep 22 '17 at 10:14
My packaging was also jar and I also need to do this – Sergio Gabari Sep 04 '18 at 08:48

score 4 · Answer 8 · answered Jun 30 '19 at 10:55

For me it was a missing executions section

            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>

The failsafe with TestNG documentation at https://maven.apache.org/surefire/maven-failsafe-plugin/examples/testng.html had shown a pom without it, and it didn't work.

<plugins>
    [...]
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>3.0.0-M3</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>
    [...]
</plugins>

Adding the missing executions section shown above solved the problem.

Of course we do not write all such details in the documenttaion because the specification related to the plugin goals are phase are related to the basic understanding of the Maven and plugin documentation and Maven basics. See the goals https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html, what defaul phase (acctualy failsafe does not have default phase but surefire has it) and then we expect that the user has a common understanding of these things. I would recommend separating the goal *integration-test* and *verify* under separate execution *id*. — tibor17, Jun 30 '20 at 14:10

Horatiu · Answer 9 · 2020-07-02T03:41:53.073

1

I'm using java 8, failsafe plugin version 2.22.2 => tests not running problem, to solve this I added next:

<plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-failsafe-plugin</artifactId>
     <version>${surefire.plugin.version}</version>
     <dependencies>
         <dependency>
             <groupId>org.apache.maven.surefire</groupId>
             <artifactId>surefire-junit47</artifactId>
             <version>2.22.2</version>
         </dependency>
    </dependencies>
     <executions>
         <execution>
             <id>integration</id>
             <phase>integration-test</phase>
             <goals>
                 <goal>integration-test</goal>
            </goals>
        </execution>
        <execution>
            <id>verify</id>
            <phase>verify</phase>
            <goals>
                <goal>verify</goal>
            </goals>
        </execution>
     </executions>
</plugin>

edited Jul 02 '20 at 03:41

answered Jun 29 '20 at 13:41

Horatiu

375
3
6

The goals **integration-test** and **verify** have to be listed in the execution(s) section of the plugin. Otherwise you won't see it running. – tibor17 Jun 30 '20 at 14:14
I added also the execution part. – Horatiu Jul 02 '20 at 03:39
1

I am highly recommending to use one version ${surefire.plugin.version} and not hardcoded version 2.22.2. – tibor17 Jul 02 '20 at 13:29

score 0 · Answer 10 · answered Sep 19 '18 at 08:01

0

I had somewhat similar issue. My Integration tests were not being picked up. My project is a multi module spring boot project. And my mistake was that I added maven-failsafe-plugin in the parent POM. After fiddling around with many options I placed the failsafe-plugin in the module pom where all my integration tests were present which solved my problem.

answered Sep 19 '18 at 08:01

horizon7

1,113
14
18

The goals **integration-test** and **verify** have to be listed in the execution(s) section of the plugin. Otherwise you won't see it running. The place: parent or child was not the answer why you made it running. – tibor17 Jun 30 '20 at 14:14

score 0 · Answer 11 · edited Mar 18 '20 at 14:46

0

For me, surefire:verify did not run any tests. I had to use surefire:integration-test.

edited Mar 18 '20 at 14:46

pcampana

2,413
2
21
39

answered Mar 18 '20 at 13:36

cliffberg

11
2

Of course it won't work because **surefire** does not have the goals **verify** and **integration-test**, see the documentation https://maven.apache.org/surefire/maven-surefire-plugin/plugin-info.html – tibor17 Jul 02 '20 at 13:23
If you want to use these goals, you have to use **failsafe** plugin, and the commands will be in this order **mvn failsafe:integration-test** and **mvn failsafe:verify**, see the goals for failsafe in documentation https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html – tibor17 Jul 02 '20 at 13:25

mcvkr · Answer 12 · 2021-01-27T19:24:49.887

Since there are so many pieces that can lead to this problem, I will share how mine was resolved.

There was an active profile in my maven settings file ( .m2/settings.xml ), it had maven.test.skip property set to true and that was skipping the tests when I run the tests via maven. It was needed for an old application and I totally forgot about that profile.

</profiles>
   <profile>
      <id>god-damnit-profile</id>
      <properties>
        ...
        <maven.test.skip>true</maven.test.skip>
      </properties>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>god-damnit-profile</activeProfile>
  </activeProfiles>

I commented it out and it works now.

<activeProfiles>
    <!-- <activeProfile>god-damnit-profile</activeProfile> -->     
</activeProfiles>

score 0 · Answer 13 · answered Nov 03 '21 at 11:28

0

If you use JUnit5, don't forget to add the following dependency to your test classpath:

    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <scope>test</scope>
    </dependency>

answered Nov 03 '21 at 11:28

kolobok

3,835
3
38
54

Maven fail-safe not executing tests

13 Answers13

Linked