9

I’m using Maven 3.2.3, JUnit 4.12, and SureFire 2.18.1 within Jenkins. I have a multi-module Maven job set up with the following Maven options:

clean install -U -e -P qa -Dci -DskipITs

However, when I run my Maven build, although one of my JUnit tests fail, the Maven module itself reports "SUCCESS" (output after) ... 

Executing Maven:  -B -f /var/lib/jenkins/workspace/subco/subco/pom.xml clean install -U -e -P qa -Dci -DskipITs


…

———————————————————————————
 T E S T S
-------------------------------------------------------
Running org.mainco.subco.user.domain.UserTest
Tests run: 24, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec <<< FAILURE! - in org.mainco.subco.user.domain.UserTest
testHasRoleNullRoles(org.mainco.subco.user.domain.UserTest)  Time elapsed: 0.006 sec  <<< FAILURE!
java.lang.AssertionError: Failed to indicate we found right role for user.
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at 

…

[INFO] Tests are skipped.
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ othersubmodule ---
[INFO] Installing /var/lib/jenkins/workspace/subco/subco/othersubmodule/target/othersubmodule.war to /var/lib/jenkins/.m2/repository/org/mainco/subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.war
[INFO] Installing /var/lib/jenkins/workspace/subco/subco/othersubmodule/pom.xml to /var/lib/jenkins/.m2/repository/org/mainco/subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] subco ....................................... SUCCESS [4.849s]
[INFO] database .......................................... SUCCESS [18.340s]
[INFO] session ........................................... SUCCESS [6.601s]
[INFO] main .............................................. SUCCESS [8:00.887s]
[INFO] mainclient ........................................ SUCCESS [4:01.490s]
[INFO] orders ............................................ SUCCESS [50.831s]
[INFO] SBD myproject Webapp .................................. SUCCESS [1:45.484s]
[INFO] subco Admin Module .......................... SUCCESS [1:01.318s]
[INFO] Other Submodule ................................ SUCCESS [6.162s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:20.351s
[INFO] Finished at: Thu Aug 20 21:17:57 UTC 2015
[INFO] Final Memory: 76M/748M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orgsclient/pom.xml to org.mainco.subco/orgsclient/83.0.0-SNAPSHOT/orgsclient-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orgsclient/target/orgsclient.war to org.mainco.subco/orgsclient/83.0.0-SNAPSHOT/orgsclient-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/pom.xml to org.mainco.subco/subco/83.0.0-SNAPSHOT/subco-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/myproject/pom.xml to org.mainco.subco/myproject/83.0.0-SNAPSHOT/myproject-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/myproject/target/myproject.war to org.mainco.subco/myproject/83.0.0-SNAPSHOT/myproject-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/database/pom.xml to org.mainco.subco/database/83.0.0-SNAPSHOT/database-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/database/target/database-83.0.0-SNAPSHOT.jar to org.mainco.subco/database/83.0.0-SNAPSHOT/database-83.0.0-SNAPSHOT.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/othersubmodule/pom.xml to org.mainco.subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/othersubmodule/target/othersubmodule.war to org.mainco.subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/sbadmin/pom.xml to org.mainco.subco/sbadmin/83.0.0-SNAPSHOT/sbadmin-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/sbadmin/target/sbadmin.war to org.mainco.subco/sbadmin/83.0.0-SNAPSHOT/sbadmin-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/pom.xml to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/target/main.jar to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/target/main-tests.jar to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT-tests.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orders/pom.xml to org.mainco.subco/orders/83.0.0-SNAPSHOT/orders-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orders/target/orders.jar to org.mainco.subco/orders/83.0.0-SNAPSHOT/orders-83.0.0-SNAPSHOT.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/session/pom.xml to org.mainco.subco/session/83.0.0-SNAPSHOT/session-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/session/target/session.jar to org.mainco.subco/session/83.0.0-SNAPSHOT/session-83.0.0-SNAPSHOT.jar
Sending e-mails to: sb-engineers@evotext.com laura.curry@evotext.com deb.arsenault@evotext.com bob.bean@evotext.com laredotornado@live.com

channel stopped Sending e-mails to: list@myco.com Warning: you have no plugins providing access control for builds, so falling back to legacy behavior of permitting any downstream builds to be triggered Finished: UNSTABLE

How do I get the Maven module to report FAILURE if one of the unit tests fails? I realize the overall build reports as UNSTABLE in Jenkins, but for readability, i would prefer SUCCESS to show only if everything passed.

Edit: Here is the surefire plugin configuration

                    <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-surefire-plugin</artifactId>
                            <version>2.18.1</version>
                            <configuration>
                                    <reuseForks>true</reuseForks>
                                    <argLine>-Xmx2048m -XX:MaxPermSize=512M -noverify -XX:-UseSplitVerifier ${argLine}</argLine>
                                    <skipTests>${skipAllTests}</skipTests>
                            </configuration>
                    </plugin>
mainframer
  • 20,411
  • 12
  • 49
  • 68
Dave
  • 15,639
  • 133
  • 442
  • 830
  • So you get Unstable and you want Failure, is that correct? – Zloj Aug 21 '15 at 13:44
  • 1
    I think he wants maven to fail the build instead of showing success. Out of curiosity, have you tried manually setting the test failure ignore (http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#testFailureIgnore) property to false? – Nick DeFazio Aug 21 '15 at 13:46
  • Hi, I want what spoonybard896 described -- the Maven build to fail instead of showing success. I haven't set the property you mention explicitly. Including my surefire config in case that adds any clarity to this. – Dave Aug 21 '15 at 14:10
  • It's difficult to tell without seeing your whole setup. Have you actually verified that the maven.test.failure.ignore user property is false? – sandris Sep 07 '15 at 09:03
  • This can only be set by the plugin, right? Is there a way to output what this value is as Maven runs, (e.g. adding "-X" or somethign similar)? – Dave Sep 07 '15 at 20:53
  • Possible duplicate of [How do I make Jenkins build fail when Maven unit tests fail?](https://stackoverflow.com/questions/28683518/how-do-i-make-jenkins-build-fail-when-maven-unit-tests-fail) – eis Jun 12 '18 at 10:07

1 Answers1

12

This answer is basically what @spoonybard896 suggested in a comment.

Add -Dmaven.test.failure.ignore=false to MAVEN_OPTS. That can be added in the Build -> Advanced section of your Jenkins Job.

Normally this should be the default value, but with Jenkins and Maven jobs, it isn't so. For those, this property is actually set to true unless you set it yourself.

(disclaimer: looked up the exact Jenkins steps from this answer).

Community
  • 1
  • 1
eis
  • 51,991
  • 13
  • 150
  • 199