Using JUnit on code that terminates

Question

I am trying to test a given java application, and for that purpose I want to use JUnit.

The problem I am facing is the following: Once the code I am trying to test finishes its work, its calling System.exit(), which closes the application. Although it is also stoping my tests from completing, as it closes the JVM (I assume).

Is there anyway to go around this problem, without modifying the original code? Initially I tried launching the application im testing from new thread, although that obviously didn't make much difference.

A Java **Application** may call System.exit(), a Java **Library** should not! — Aubin, Apr 13 '13 at 17:01
JUnit is best suited to testing library classes which take a predictable type of data as their input, and return a predictable type of data as their output. A library class should never call `System.exit()` because that makes the library harmful to applications which might use it. It sounds like you're trying to use JUnit to test non-library classes or methods, and this is not a good fit. — Bobulous, Apr 13 '13 at 17:02
My mistake its a java application: http://cs.gmu.edu/~eclab/projects/mason/ — Giannis, Apr 13 '13 at 17:02
@Arkanon is there a framework providing automated testing that is more suitable for testing applications? — Giannis, Apr 13 '13 at 17:05
Have you tried adding a shutdown hook: http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread) If you don't want to modify the application then you could add a thread from Junit as a system hook and then at least 'handle' the case when it shuts down the whole JVM (which is, as mentioned, kinda strange) — Bizmarck, Apr 13 '13 at 17:08
@Giannis the question is: why your application calls `System.exit()` inside a method? — Luiggi Mendoza, Apr 13 '13 at 17:08
@LuiggiMendoza its not my application, its a Java simulation application. Once the simulation is complete its calling System.exit(0). I can modify the code to avoid this although I would prefer to know if there is a way to do it without modifications on code. — Giannis, Apr 13 '13 at 17:10
Then the problem is your application, not JUnit related. By the way, **no sane application calls `System.exit(0)` instead will find a way to go back until the `main` method and exit cleanly. — Luiggi Mendoza, Apr 13 '13 at 17:12
There are programs available that are called acceptance tests. (Such as Fitnesse) which allows you test the application does what you want it to. As has been brought out JUnit (Unit tests) tests classes, Acceptance tests test applications and functionality. — Robert Snyder, Apr 13 '13 at 17:35

acdcjunior · Accepted Answer · 2013-04-15T14:12:48.103

You can use System Rules: "A collection of JUnit rules for testing code which uses java.lang.System."

Among their rules, you have ExpectedSystemExit, below is an example on how to use it. I believe it is a very clean solution.

import org.junit.Rule;
import org.junit.Test;
import org.junit.contrib.java.lang.system.Assertion;
import org.junit.contrib.java.lang.system.ExpectedSystemExit;

public class SystemExitTest {
    @Rule
    public final ExpectedSystemExit exit = ExpectedSystemExit.none();

    @Test
    public void noSystemExit() {
        //passes
    }

    @Test
    public void executeSomeCodeAFTERsystemExit() {
        System.out.println("This is executed before everything.");
        exit.expectSystemExit();
        exit.checkAssertionAfterwards(new Assertion() {
            @Override
            public void checkAssertion() throws Exception {
                System.out.println("This is executed AFTER System.exit()"+
                " and, if exists, the @org.junit.After annotated method!");
            }
        });
        System.out.println("This is executed right before System.exit().");
        System.exit(0);
        System.out.println("This is NEVER executed.");
    }

    @Test
    public void systemExitWithArbitraryStatusCode() {
        exit.expectSystemExit();
        System.exit(0);
    }

    @Test
    public void systemExitWithSelectedStatusCode0() {
        exit.expectSystemExitWithStatus(0);
        System.exit(0);
    }

    @Test
    public void failSystemExit() {
        exit.expectSystemExit();
        //System.exit(0);
    }

}

If you use maven, you can add this to your pom.xml:

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.11</version>
</dependency>
<dependency>
    <groupId>com.github.stefanbirkner</groupId>
    <artifactId>system-rules</artifactId>
    <version>1.3.0</version>
</dependency>

I am not sure if I use it correct but the above code does not do anything. I have exit.expectSystemExit() before the call to the class calling System.exit(0) and the application is still closing. My aim is to prevent the application from shutting down everything (including the JUnit tests). — Giannis, Apr 15 '13 at 00:37
@Giannis I see, you want to execute some code **after** `System.exit()` is called, right? I edited the answer and added an example (see `executeSomeCodeAFTERsystemExit()` method) for that. Still the cleanest solution to me. — acdcjunior, Apr 15 '13 at 04:08
@Giannis I must point out, using that solution is optimal as the code `System.out.println("This is executed AFTER System.exit()!");` is **only** executed **if** `System.exit(0)` is called! Also, this is a solution within JUnit, and it does not change any (system-wide) behaviour outside it. — acdcjunior, Apr 15 '13 at 04:11
@Giannis Did you look at the System (console) output? Did it output anything at all? — acdcjunior, Apr 15 '13 at 13:44
I have made it work. I was missing the @Rule tag. Also one important note: code in checkAssertionAfterwards() is executed after the methods annotated with After. — Giannis, Apr 15 '13 at 13:59

score 3 · Answer 2 · answered Apr 14 '13 at 01:01

System.exit(status) actually delegates the call to Runtime class. Runtime before proceeding with this shutdown request invokes checkExit(status) on JVM's current SecurityManager which can prevent the impending shutdown by throwing a SecurityException.

Usually, the SecurityManager needs to establish if the current thread has the privilege to shutdown defined by the current security policy in place but since all we need is to recover from this exit call we simply throw a SecurityException that we'll now have to catch in our JUnit test case.

In your JUnit test class, setup a SecurityManager in setUP() method:

    securityManager = System.getSecurityManager();
    System.setSecurityManager(new SecurityManager() {
        @Override
        public void checkExit(int status) {
            super.checkExit(status); // This is IMPORTANT!
            throw new SecurityException("Overriding shutdown...");
        }
    });

In tearDown() replace the SecurityManager again with the instance that we saved before. Failure to do so would prevent JUnit from shutting down now! :)

References:
http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/SecurityManager.html
http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/SecurityManager.html#checkExit(int)

The SecurityManager class contains many methods with names that begin with the word check. These methods are called by various methods in the Java libraries before those methods perform certain potentially sensitive operations. The invocation of such a check method typically looks like this:

     SecurityManager security = System.getSecurityManager();
     if (security != null) {
         security.checkXXX(argument,  . . . );
     }

The security manager is thereby given an opportunity to prevent completion of the operation by throwing an exception. A security manager routine simply returns if the operation is permitted, but throws a SecurityException if the operation is not permitted.

Thanks a lot for your great answer. I will be testing it soon. — Giannis, Apr 14 '13 at 02:22
Had to add (expected = SecurityException.class) before tests and its working perfectly. — Giannis, Apr 14 '13 at 15:41
Yes, instead of catching it `@Test(expected=SecurityException.class)` makes perfect sense here. Thanks, for accepting the answer. — Ravi K Thapliyal, Apr 14 '13 at 16:12

score -3 · Answer 3 · answered Apr 13 '13 at 17:26

-3

There is no way around System.exit() except for calling the application to run as a seperate proces (outside your JVM).

You can do this from your unit test and observe the errorlevel that comes back from it. Whether that gives enough feedback on passing of the test is up to your judgement.

answered Apr 13 '13 at 17:26

bluevoid

1,274
13
29

Maybe correct, but I am not sure. I think you can mock static and final methods with powermock. Haven't tried it so I don't know for sure. – emory Apr 14 '13 at 01:04
1

@bluevoid Not true, you can use a SecurityManager as stated above and in the duplicate questions. – Matthew Farwell Apr 14 '13 at 09:15
2

ok, stand correct. I did not know that one can intercept that call. – bluevoid Apr 15 '13 at 10:30

Using JUnit on code that terminates

3 Answers3