How do I execute a suite of TestNG integration tests contained in a side test-jar?

Question

I am trying to use TestNG on the command line in order to perform a suite of integration tests against a local development server, where test-jar-with-dependencies.jar contains the integration tests to be executed.

However, it looks like none of the integration tests in src/integration-test/ are being executed.

$ java -classpath ".;testng-6.8.8.jar;jcommander-1.27.jar;coolthing.diagnostic-5.0-SNAPSHOT-test-jar-with-dependencies.jar" org.testng.TestNG testng.xml

=========================================
diagnostic-suite
Total tests run: 0, Failures: 0, Skips: 0
=========================================

Since the maven-jar-plugin does not appear to package test dependencies, I opted to assemble a test jar using the maven-assembly-plugin where the assembly descriptor is defined as follows:

<?xml version="1.0" encoding="UTF-8"?>
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
    <id>test-jar-with-dependencies</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <useProjectArtifact>true</useProjectArtifact>
            <useProjectAttachments>true</useProjectAttachments>
            <unpack>true</unpack>
            <scope>test</scope>
        </dependencySet>
    </dependencySets>
</assembly>

In order to perform assembly as part of the Maven install phase I attached the assembly descriptor as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    ...
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>test-jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptors>
                        <descriptor>src/integration-test/resources/test-jar-with-dependencies.xml</descriptor>
                    </descriptors>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

This part seems to work perfectly, except I am unable to execute the integration tests contained within the test JAR. The TestNG suite XML file is defined as follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="diagnostic-suite" parallel="classes" thread-count="4">
    <test name="endpoints">
        <groups>
            <dependencies>
                 <group depends-on="ping" name="diagnostic"></group>
            </dependencies>
            <run>
                <include name="ping" />
                <include name="diagnostic" />
            </run>
        </groups>

        <classes>
            <class name="our.company.ping.CoolThingPingIT" />
            <class name="our.company.status.CoolThingIndexIT" />
            <class name="our.company.status.CoolThingConfigurationIT" />
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Can you think of where I may have missed a step? The expected result is that the tests defined within the TestNG suite will be executed, however none are executed.

score 0 · Accepted Answer · answered Sep 23 '15 at 11:27

Try using the testjar and xmlpath options ..

java -cp MyProject-jar-with-dependencies.jar;MyProject.jar;MyProject-tests.jar org.testng.TestNG -testjar MyProject-tests.jar -xmlpathinjar suites/GroupBased_Tests.xml

I had similar requirement documented here

score 0 · Answer 2 · edited May 23 '17 at 12:01

In terms of packing the test-jar, here's what I have:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <executions>
        <execution>
            <id>attach-tests</id>
            <phase>package</phase>
            <goals>
                <!-- Always build a test-jar (if test classes present) -->
                <goal>test-jar</goal>
            </goals>
        </execution>
    </executions>
</plugin>

In terms of running tests from the jar, check out this question and what I have posted here too

I have marked the parts that are relevant to you (the dependencies part and then the dependenciesToScan part):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>test</groupId>
  <artifactId>test-runner</artifactId>
  <version>1.0.0-SNAPSHOT</version>

  <dependencies> <!-- <<< dependencies, incl. the "tests" dependency -->
    <dependency>
      <groupId>${test.library.groupId}</groupId>
      <artifactId>${test.library.artifactId}</artifactId>
      <version>${test.library.version}</version>
    </dependency>
    <dependency>
      <groupId>${test.library.groupId}</groupId>
      <artifactId>${test.library.artifactId}</artifactId>
      <version>${test.library.version}</version>
      <classifier>tests</classifier>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.18</version>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>${test.suite}.xml</suiteXmlFile> <!-- <<< this is the issue -->
          </suiteXmlFiles>
          <dependenciesToScan> <!-- <<< this is what makes you find the tests within the jar -->
            <dependency>${test.library.groupId}:${test.library.artifactId}</dependency>
          </dependenciesToScan>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

How do I execute a suite of TestNG integration tests contained in a side test-jar?

2 Answers2