13

I am implementing some tests for an existing Java Swing application, so that I can safely refactor and extend the code without breaking anything. I started with some unit tests in JUnit, since that seems the simplest way to get started, but now my priority is to create some end-to-end tests to exercise the application as a whole.

I am starting the application afresh in each test by putting each test method in a separate test case, and using the fork="yes" option in Ant's junit task. However, some of the use cases I would like to implement as tests involve the user exiting the application, which results in one of the methods calling System.exit(0). This is regarded by JUnit as an error: junit.framework.AssertionFailedError: Forked Java VM exited abnormally.

Is there a way to tell JUnit that exiting with a return code of zero is actually OK?

Ben
  • 2,348
  • 2
  • 19
  • 28
  • ###See also > [Java: How to test methods that call System.exit()?](http://stackoverflow.com/questions/309396/java-how-to-test-methods-that-call-system-exit) – Stefan Birkner Dec 28 '11 at 16:41

4 Answers4

33

The library System Rules has a JUnit rule called ExpectedSystemExit. With this rule you are able to test code, that calls System.exit(...):

public class MyTest {
    @Rule
    public final ExpectedSystemExit exit = ExpectedSystemExit.none();

    @Test
    public void systemExitWithArbitraryStatusCode() {
        exit.expectSystemExit();
        /* the code under test, which calls System.exit(...)
         * with an arbitrary status
         */
    }

    @Test
    public void systemExitWithSelectedStatusCode0() {
        exit.expectSystemExitWithStatus(0);
        //the code under test, which calls System.exit(0)
    }
}

System Rules needs at least JUnit 4.9.

Full disclosure: I'm the author of System Rules.

bric3
  • 40,072
  • 9
  • 91
  • 111
Stefan Birkner
  • 24,059
  • 12
  • 57
  • 72
  • 1
    Very nice! System Rules is great for testing command-line tools that often do things like System.exit(), write to System.out and so on. – Torsten Römer Sep 24 '14 at 11:32
  • 1
    Cool stuff. I misleed by the imports (org.junit.contrib.java.lang.system.ExpectedSystemExit;) in some examples. It's not in https://mvnrepository.com/artifact/org.junit.contrib but in the https://mvnrepository.com/artifact/com.github.stefanbirkner/system-rules – kecso Jul 01 '16 at 18:37
8

How I deal with that is to install a security manager that throws an exception when System.exit is called. Then there is code that catches the exception and doesn't fail the test.

public class NoExitSecurityManager
    extends java.rmi.RMISecurityManager
{
    private final SecurityManager parent;

    public NoExitSecurityManager(final SecurityManager manager)
    {
        parent = manager;
    }

    public void checkExit(int status)
    {
        throw new AttemptToExitException(status);
    }

    public void checkPermission(Permission perm)
    {
    }
}

And then in the code, something like:

catch(final Throwable ex)
{
    final Throwable cause;

    if(ex.getCause() == null)
    {
        cause = ex;
    }
    else
    {
        cause = ex.getCause();
    }

    if(cause instanceof AttemptToExitException)
    {
        status = ((AttemptToExitException)cause).getStatus();
    }
    else
    {
        throw cause;
    }
}

assertEquals("System.exit must be called with the value of " + expectedStatus, expectedStatus, status);
TofuBeer
  • 60,850
  • 18
  • 118
  • 163
  • That works, and FEST-Swing even provides its own NoExitSecurityManager. I think this will be my approach for now, since it involves no changes to existing application code. The only downside is that the application has an UncaughtExceptionHandler that pops up a bug-reporting dialog when the security manager blocks the exit call. The test still passes, but anyone watching the test might get the impression the application has crashed. – Ben May 26 '11 at 16:49
6

Could you abstract out the "system exiting" into a new dependency, so that in your tests you could just have a fake which records the fact that exit has been called (and the value), but use an implementation which calls System.exit in the real application?

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • That is a possibility. I would like to get as much as possible of the test suite in place _before_ making any changes, but I guess sometimes you just have to dive in and make a few careful changes to get the tests working. – Ben May 26 '11 at 16:46
  • @Ben: Hopefully in this case it would be a relatively small change - and I'd definitely prefer to do this than mess around with the security manager. Of course, you'll also need to make sure that the app's Swing thread terminates appropriately etc. – Jon Skeet May 26 '11 at 16:47
  • Looking at the code, it might be a bit more involved than I have time for just yet, but undoubtedly a good exercise in applying the lessons from "Working Effectively with Legacy Code". I'm not sure how to get the Swing thread to terminate appropriately: just dispose of the open Swing windows? It seems safer at this stage to fork a new VM for each test, which is why I am restricted to one test per class. I'm glad to receive better suggestions though. – Ben May 27 '11 at 10:28
  • @Ben: I believe that closing all the open windows will do it, yes. A long time ago it didn't, but I believe it does now. – Jon Skeet May 27 '11 at 10:43
4

If anybody needs this functionality for JUnit 5, I've written an extension to do this. This is a simple annotation you can use to tell your test case to expect and exit status code or a specific exit status code.

For example, any exit code will do:

public class MyTestCases { 

    @Test
    @ExpectSystemExit
    public void thatSystemExitIsCalled() {
        System.exit(1);
    }
}

If we want to look for a specific code:

public class MyTestCases {

    @Test
    @ExpectSystemExitWithStatus(1)
    public void thatSystemExitIsCalled() {
        System.exit(1);
    }
}
Todd
  • 30,472
  • 11
  • 81
  • 89