Our Maven build has lots of unit tests in "src/test/java" (named "*Test.java") that work fine from Eclipse and from the maven build.
We also have a bunch of integration tests in "src/test/java" (named "*IT.java") that work fine from Eclipse, but we don't run them from Maven, as the nature of the tests make them unreliable except for manual use.
I've now written one that will be reliable, and which I want to execute from the mvn command line in a CI build. As of yet, I can't figure out how to get Failsafe to execute it.
I noticed the advice in a previous SO post and made sure I followed those guidelines (I was already using the correct location and naming pattern), but I still get "Tests run: 0" when I run "failsafe:integration-test".
The following is an elided version of the pom:
<?xml version="1.0"?>
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<parent>
...
</parent>
<artifactId>usl-features-install</artifactId>
<name>usl-features-install</name>
<url>http://maven.apache.org</url>
<packaging>bundle</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>attach-artifacts</id>
<phase>package</phase>
<goals>
<goal>attach-artifact</goal>
</goals>
<configuration>
<artifacts>
...
</artifacts>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
When I run "mvn failsafe:integration-test", I see the following:
[INFO] ------------------------------------------------------------------------
[INFO] Building usl-features-install 3.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-cli) @ usl-features-install ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Update:
Referring to the provided answer, I initially added the following executions block (and version):
<version>2.20.1</version>
<executions>
<execution>
<goals>
<!-- <goal>verify</goal> -->
<goal>integration-test</goal>
</goals>
</execution>
</executions>
and then running "mvn failsafe:integration-test". This had no effect.
I then removed the comment wrappers for "verify" above and ran "mvn verify". The results from that were very confusing. Running "failsafe:integration-test" did the same as before, but running "mvn verify" did the following:
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default) @ usl-features-install ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-features-install ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.968 s
[INFO] Finished at: 2017-10-25T10:41:26-07:00
[INFO] Final Memory: 35M/498M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify (default) on project usl-features-install: There are test failures.
[ERROR]
[ERROR] Please refer to C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\failsafe-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
Notice that it reports on failsafe twice. The first time still doing nothing, and the second one failing.
The mentioned "dump file" had the following stacktrace, confusing the matter more:
java.lang.NoSuchMethodError: org.apache.maven.surefire.report.RunListener.testSetStarting(Lorg/apache/maven/surefire/report/ReportEntry;)V
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:235)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)
Update:
I tried running mvn with "-X". It of course generated a lot of output, but here's what it showed as it was attempting to run the it tests:
[DEBUG] Forking command line: cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_102\jre\bin\java" -javaagent:C:\\Users\\<myuid>\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.7.8\\org.jacoco.agent-0.7.8-runtime.jar=destfile=C:\\users\\<myuid>\\git\\oce_usl\\usl-parent\\usl-features-install\\target\\jacoco.exec -jar C:\cygwin64\tmp\surefire29734070406705707\surefirebooter6462732870353032119.jar C:\cygwin64\tmp\surefire29734070406705707 2017-10-25T15-17-49_742-jvmRun1 surefire1801590010194716332tmp surefire_05587249170077752672tmp"
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-features-install ---
[DEBUG] Configuring mojo org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify from plugin realm ClassRealm[plugin>org.apache.maven.plugins:maven-failsafe-plugin:2.20.1, parent: sun.misc.Launcher$AppClassLoader@5c647e05]
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify' with basic configurator -->
[DEBUG] (s) basedir = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install
[DEBUG] (s) reportsDirectory = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\surefire-reports
[DEBUG] (s) skip = false
[DEBUG] (f) summaryFile = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\failsafe-reports\failsafe-summary.xml
[DEBUG] (s) testClassesDirectory = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\test-classes
[DEBUG] (s) testFailureIgnore = false
[DEBUG] (f) session = org.apache.maven.execution.MavenSession@6bf13698
[DEBUG] -- end configuration --
[DEBUG] Failsafe report directory: C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\surefire-reports
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
This shows that it's using the jacoco agent that I need for running unit tests. I don't need it for integration tests. I'm not sure how I would turn that off, or whether that's even a factor here.
Update:
I think I've arrived at a "fix", or at least a medium-term workaround.
I felt that the "NoSuchMethodError" issue was probably key to this, so I googled for a similar occurrence, and found the following issue for surefire: https://github.com/junit-team/junit5/issues/809 .
So, what I did was change my failsafe version from 2.20.1 to 2.19.1 (which apparently causes the same version of surefire to be used), and now it doesn't fail in that way.
However, I'm now seeing another occasional problem with "Command NOOP unexpectedly read Void data with length 4. " (http://maven.40175.n5.nabble.com/surefire-forked-vm-failed-td5858900.html), which appears to engender a suggestion to back off even further, to surefire 2.18.1.
This is compounded with the fact that this test is a PaxExam test, which presents other complications and messes. I think I'm heading to a conclusion that the fix is worse than the problem.
