How to test a java console application with Maven?

Question

These tests should run after the jar file is built. They should test whether it runs at all, and whether it produces correct console output given certain input files.

Could you point at some examples of such tests for console applications? Do these test have to be written in Java to be used with Maven?

There is a similar question (Testing console based applications/programs - Java) but what I need is blackbox testing, not unit testing. And it does not mention Maven.

UPDATE:

It turned out that it vas very easy to do. My first problem was a misconception of integration tests after seeing so many definitions here (What is Unit test, Integration Test, Smoke test, Regression Test?; What's the difference between unit, functional, acceptance, and integration tests?; What are unit testing and integration testing, and what other types of testing should I know about?; What is an integration test exactly?). The second problem was that I did not intend to write these test in Java, but in the end I had to learn how to use java.lang.Runtime.

First of all, I have added this to my pom.xml:

<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.6</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Then I have created an *IT.java file in src/test/java:

public class MyAppIT {

    @Test
    public void test() throws IOException {
        Runtime runtime = Runtime.getRuntime();
        String cmd = "java -cp target/myapp.jar my.app.Main";
        Process process = runtime.exec(cmd);
        InputStreamReader isr = new InputStreamReader(process.getErrorStream());
        BufferedReader br = new BufferedReader(isr);
        Boolean containsUsage = false;
        String line;
        while ((line = br.readLine()) != null) {
            if (line.contains("Usage")) {
                containsUsage = true;
            }
        }
        assertTrue(containsUsage);
    }
}

Now I use mvn verify instead of mvn package.

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

burna · Accepted Answer · 2012-12-28T12:53:34.283

It seems you want integration testing.

In many cases, the application is built by a continuous integration (CI) system like Hudson, Travis or sth. This CI compiles and runs the unit-tests after each commit to the sourcecode. After successful compilation, the integration-test projects are triggered.

The integration-tests reside in a separate maven source project. You would create the integration-test-classes like normal unit tests in src/test/java. Of course you don't test a single class in isolation - that is unit testing - you setup your system for integration-tests (wire the right components) and execute the needed tests "smoke test".

So, in your case, you could create another project for integration-test and execute this after the build of your application. You could create a maven parent-pom and use sub-modules to avoid the manual integration-test-build.

The test classes need to be java and located in src/test/java, they look like normal unit tests and are run by maven. Of course you could start a non-java program in that tests and assert the outcome.

A console app has a "main" method, so I would call the main method in the test. Otherwise you have to run a shell command or something. (via java.lang.Runtime)

There is no speciality in testing console apps. Unless you want to test the user interface - automated testing of user interfaces is not easy and more difficult, please clarify if this is your focus.

Yes, what I want is to test if the program actually runs, not if the "**main**" method works. I would not call it interface testing for my program does not allow any user interaction. It just processes input files and outputs results. — utapyngo, Dec 28 '12 at 13:09
The command-line call issues a call to the main method. You could use Runtime to exec a "java my.app.Main" command, you have the full java possibilities in the test project. — burna, Dec 28 '12 at 13:18

yegor256 · Answer 2 · 2012-12-29T14:42:10.457

1

I would recommend to use maven-invoker-plugin in combination with exec-maven-plugin. Invoker makes a sub-project and runs it, where the exec plugin starts your JAR. The invoker then validates its output.

For example, take a look at this pom.xml in s3auth-relay project. The module itself produces a JAR console application, which is a HTTP daemon. The pom.xml is part of maven-invoker-plugin configuration. It starts the console application and checks whether it shuts down gracefully. To prevent the JAR to start as a daemon we're using an optional -d flag.

The example doesn't show the ability to test the output of the JAR, but it's possible to do using post build script mechanism of maven-invoker-plugin.

edited Dec 29 '12 at 14:42

answered Dec 28 '12 at 14:49

yegor256

102,010
123
446
597

Unfortunately maven-invoker-plugin is intended to run Maven as a part for integration tests for maven-plugins in particular but not a console application. – khmarbaise Dec 28 '12 at 15:16
As long as your console application doesn't require user input maven-invoker-plugin is a perfect fit – yegor256 Dec 28 '12 at 16:38
Do you have some examples of usage? – khmarbaise Dec 28 '12 at 17:01
As far as I understand, this approach can only test if the application actually runs and *shuts down gracefully*, nothing more. It is not possible to analyze its output this way. Am I right? – utapyngo Dec 29 '12 at 14:19
Yes, only if it shuts down gracefully. I updated my answer to show the ability to analyze its output. – yegor256 Dec 29 '12 at 14:42

How to test a java console application with Maven?

2 Answers2