There are a number of schools of thought as to how to handle this type of integration test with Maven.
I should point out that when you are deploying an application to an application server, you are not in the realm of unit testing any more. Because the entire application is being deployed within a container, you are testing the integration of those two components.
Now there is nothing wrong with using JUnit for running integration tests (though there are some limitations that you may hit, for example unit tests should not care about the sequencing of individual tests - assuming you are writing them correctly - so JUnit enforces this by not guaranteeing any sequence of execution... prior to Java 1.7 the execution order was accidentally implied by the order of test methods within a class, but it was not part of the JUnit contract... Some people switch to other testing frameworks for their integration tests, e.g. TestNG, if they find the unit test focus of JUnit is getting in the way of their test development)
The key point to keep in mind is that the Maven lifecycle uses the test
phase for the execution of unit tests.
When it comes to integration tests there are two (and a half) schools of thought as to the right way to handle the tests with Maven.
School 1 - Failsafe and integration-test/verify
This school of thought uses the phases after package
to start up a container, run the integration tests, tear down the container, and finally check the test results and fail the build in the event of test failures.
NEVER EVER RUN mvn integration-test
as that will not tear down the container correctly, any time you think you want to type mvn integration-test
you actually want to type mvn verify
(oh look, it's shorter and easier to type also... bonus)
So with this you do the following:
For extra brownie points you would use build-helper-maven-plugin:reserve-network-port bound to the validate
phase to ensure that the test server is started on an unused network port and then either use resource filtering against the test resources to pass the port through to the tests or use a system property passed through systemPropertyVariables to make the port number available to the tests.
Advantages
- Clean Maven build
- If the tests fail, you cannot release the project
- Can move the integration tests into a separate profile (by convention called
run-its
) if the tests are too slow to run every build.
Disadvantages
- Hard to run the tests from an IDE. All the integration tests start/end in
IT
and while Maven knows to run tests starting/ending in Test
with Surefire and run tests starting/ending in IT
with Failsafe, your IDE probably doesn't. Additionally, your IDE is not going to start the container for you, so you have to do a lot of work by hand to actually run the tests manually.
Debugging the tests potentially requires attaching two debuggers, e.g. one to debug the application running in container and the other to debug the test cases.
mvnDebug -Dmaven.failsafe.debug=true verify
Couples your tests to the Maven build process.
School 2 - Separate module
This school of thought moves the integration tests into a separate module that depends on the war
module and copies the war
into the test resources using, e.g. dependency:copy-dependencies
bound to the generate-test-resources
phase coupled with a Tomcat7 dependency to test against.
The test cases themselves start up the Tomcat7 container using embedded mode
Advantages
- Tests can run in IDE
- Integration tests are separated from Unit tests, so asking the IDE to run all tests will not kick off the slower tests
Disadvantages
- The
war
artifact is only rebuilt if you go past the package
phase, consequently, you need to run at least mvn clean package
periodically to refresh the code under test when using the IDE.
- The failure of the integration tests does not break the build of the
war
module, so you can end up releasing a broken war
artifact and then have the reactor build fail for the integration test module. Some people counteract this issue by having the integration test module within src/it
and using Maven Invoker Plugin to run the tests... though that provides a poorer IDE integration, so I do not recommend that line.
- Hard to get a consolidated test coverage report from Maven.
- Have to code the container start/stop yourself from within your test cases.
School 2.5 - Failsafe with the test cases starting their own Tomcat7 server
This is a kind of hybrid of the two approaches.
You use Failsafe to execute the tests, but the tests themselves are responsible for starting and stopping the Tomcat7 container that you want to test in.
Advantages
- Don't have to configure the server start/stop in Maven pom
- IDE can safely run all tests (though the integration tests may be slower and you may want to not run them, but it's not like they will all fail unless there is a test failure)
- Easier to debug the tests from your IDE (only one process to attach against, and the IDE usually makes it easy to debug tests by providing a special test runner)
Disadvantages
- Have to code the container start/stop yourself from within your test cases
I hope the above helps you understand the options you have. There may be other tweaks but in general the above are considered the best practice(s) for integration testing with Maven at present.