142

What is the difference between Maven Surefire and Maven Failsafe plugins?
I have searched all over web, but did not get the answer.

Jacob van Lingen
  • 8,989
  • 7
  • 48
  • 78
bugCracker
  • 3,656
  • 9
  • 37
  • 58

3 Answers3

184

In simple words, the Failsafe plugin is designed to run integration tests while Surefire to run unit tests.

This is further explained in Maven FAQ:

  • maven-surefire-plugin is designed for running unit tests and if any of the tests fail then it will fail the build immediately.

  • maven-failsafe-plugin is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests.

    The name "failsafe" was chosen both because it is a synonym of surefire and because it implies that when it fails, it does so in a safe way.

    The Failsafe Plugin has two goals:

See also:

Community
  • 1
  • 1
kenorb
  • 155,785
  • 88
  • 678
  • 743
  • 13
    The interesting question is _why_. Many build tools work perfectly without making this arguably rather artificial and limited distinction. Configuring both plugins together and absorbing their reports in consistent ways is quite the PITA. – Raphael Mar 31 '20 at 15:32
  • 1
    And no, the differences stated in the other answers do not provide sufficient reason for having multiple tools for what should be filters and flags. – Raphael Mar 31 '20 at 15:33
  • 5
    I think this answer fails to make a specific case for Failsafe. It also uses two more lifecycle phases within maven, `pre-integration-test` and `post-integration-test`. This is generally used to bring up an environment, or do some sort of external setup. If a test fails when running in failsafe, it is guaranteed that `post-integration-test` will run and tear things down. This is because, as stated in the answer, it does not fail until the verify phase. That is the real benefit of the plugin, it will ALWAYS run `post-integration-test` – Michiel Bugher Nov 02 '20 at 18:46
  • Technically, the plugin itself does not execute these phases, but it does adhere to the convention. Surefire will fail immediately when a test failure is detected and no subsequent maven lifecycle phases are executed. – Michiel Bugher Nov 02 '20 at 18:51
34

From https://maven.apache.org/surefire/maven-failsafe-plugin/, I would say that the difference between Surefire and Failsafe is the way they fail:

If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.

The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.

Community
  • 1
  • 1
Jonas Wolf
  • 1,154
  • 1
  • 12
  • 20
  • 9
    Then the question becomes: why ever use Surefire if they are telling us it can leave stuff in an incorrect state? – user118967 Apr 12 '20 at 17:22
7

In my country its the second google result when searching for "maven failsafe maven surefire" to get to this FAQ: Difference between maven-failsafe-plugin and maven-surefire-plugin which states:

maven-surefire-plugin is designed for running unit tests and if any of the tests fail then it will fail the build immediately.

maven-failsafe-plugin is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests."

Sniper
  • 1,428
  • 1
  • 12
  • 28
JBA
  • 2,769
  • 5
  • 24
  • 40