4

I'm setting up a CD pipeline in Jenkins and I want to run my unit tests and integration tests in two different steps. The plan is to have my pipeline script look something like this and have them run separately:

    stage('Unit tests') {
        steps {
            withMaven(maven: 'Maven 3.6.2') {
                sh 'mvn test -P coverage'
            }
        }
    }
    stage('Integration tests') {
        steps {
            withMaven(maven: 'Maven 3.6.2') {
                sh 'mvn test -P coverage'
            }
        }

I have tried using the surefire plugin as described here: https://dzone.com/articles/splitting-unit-and-integration-tests-using-maven-a, and running 'mvn test' does run only the unit test as it should, but 'mvn integration-test' runs both unit and integration tests.

I have also tried using the failsafe plugin as described here: Maven separate Unit Test and Integration Tests, but 'mvn verify' runs both unit and integration tests no matter which options I enter.

How can I make my pipeline execute the unit tests and integration tests in two different steps?

Pom with surefire:

            <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId> <!-- surefire plugin version managed by Spring Boot -->
            <configuration>
                <skipTests>true</skipTests>
            </configuration>
            <executions>
                <execution>
                    <id>unit-tests</id>
                    <phase>test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <skipTests>false</skipTests>
                        <includes>
                            <include>**/*Test.java</include>
                        </includes>
                    </configuration>
                </execution>
                <execution>
                    <id>integration-tests</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <skipTests>false</skipTests>
                        <includes>
                            <include>**/*IT.*</include>
                            <include>**/*Tests.*</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>

Pom with surefire and failsafe:

            <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.19.1</version>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <version>2.19.1</version>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.1</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <skip>${surefire.skip}</skip>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
PalBo
  • 2,203
  • 3
  • 22
  • 43
  • Could you provide example pom.xml or better an example project ( with example code ) ? – Vokail Oct 16 '19 at 12:50
  • 1
    Why do you like to separate them? Running units should be done always...sometimes it might needed to have running integration tests separately usually done via maven-failsafe-plugin and using `mvn clean verify -Prun-its` (profile only if your integration tests take very long time) ...Do not use maven-surefire-plugin to run integration tests...follow convention and use unit tests name `*Test.java` whereas integration tests name `*IT.java`(details in the docs) ...do not configure surefire in the way you are doing cause there existing very good defaults...which is in 99.999% of the cases correct. – khmarbaise Oct 16 '19 at 13:04
  • Both unit and integration tests will be run every build, but we would like to separate them to have better control of where things fail. – PalBo Oct 16 '19 at 13:06
  • As I wrote ...using naming conventions correctly and you can control via `mvn clean test` running unit tests and via `mvn clean verfiy`(may be supplemental `-Prun-its` to run also integration tests.. – khmarbaise Oct 16 '19 at 13:07
  • Do not bind maven-surefire-plugin to integration-test part this is intended with maven-failsafe-plugin ... – khmarbaise Oct 16 '19 at 13:11
  • Okay. So I did as you recommended @khmarbaise, my pom now includes the surefire and failsafe plugin as they are by default, I have included nothing extra. Running `mvn clean test` now runs only unit tests, but `mvn clean verify` runs both. How can I make `verify` run only integration tests? – PalBo Oct 16 '19 at 13:17
  • What is the problem with running unit tests? – khmarbaise Oct 16 '19 at 13:24
  • It's not needed to run them twice, like I said during our build we run `mvn clean test` which should perform only unit tests, and directly after `mvn clean verify` which should run only integration tests. As it is now, the unit tests will be run both during `test` and `verify` – PalBo Oct 16 '19 at 13:31
  • Couldn't you run the failsafe plugin separately? – J Fabian Meier Oct 16 '19 at 14:00
  • My question is: Why would you like to separate those things? And why would like to run the whole process two times? Cause each time you run `mvn clean test` or `mvn clean verify` things like compiling etc. is done two time? So I don't see the advantage of preventing running the unit tests? – khmarbaise Oct 16 '19 at 17:52

1 Answers1

3

This solved it for me:

Below is example pom.xml which works with the following commands:

  • mvn clean verify -DskipUTs=true : Runs only integration tests (tests ending in "IT")
  • mvn clean verify -DskipITs=true : Runs only unit tests (tests ending in "Test")

  • mvn clean verify -DskipTests=true : Skips all tests

    <build>
<finalName>test-app</finalName>
<plugins>

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId> <!-- surefire plugin version managed by Spring Boot -->
        <configuration>
            <skipTests>${skipUTs}</skipTests>
        </configuration>
    </plugin>

    <plugin>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>3.0.0-M3</version>
        <executions>
            <execution>
                <id>run-integration-tests</id>
                <phase>integration-test</phase>
                <goals>
                    <goal>integration-test</goal>
                    <goal>verify</goal>
                </goals>
            </execution>
        </executions>
        <configuration>
            <skipTests>${skipTests}</skipTests>
            <skipITs>${skipITs}</skipITs>
        </configuration>
    </plugin>

</plugins>
</build>
PalBo
  • 2,203
  • 3
  • 22
  • 43