Can I skip a JUnit test while the test is running?

Question

Suppose I want to manually run from my IDE (Intellij IDEA, or eclipse) 4000 JUnit tests; the first 1000 tests run pretty smoothly (say they take 3 minutes all 1000) but the test 1001 takes alone over 30 minutes. Is there a way I can skip the test 1001 (while it's still running) and to let the test 1002 (and the others) keep going. I do not want to @Ignore the test 1001 and rerun the suite because I already have the answer for tests 1-1000; also I do not want to select tests 1001-4000 because it takes too much time.

I would some kind of button - Skip Current Test - which can be pressed when the test is running.

In case such feature does not exist, an enhancement for it needs to be done by the IDE developers or by JUnit developers?

Answer 1

This is actually pretty simple with JUnit 4 using Assume. Assume is a helper class like Assert. The difference is that Assert will make the test fail while Assume will skip it.

The common use case is Assume.assumeTrue( isWindows() ) for tests that only work on, say, a Windows file system.

So what you can do is define a system property skipSlowTests and add

Assume.assumeTrue( Boolean.getBoolean("skipSlowTests") )

at the beginning of slow tests that you usually want to skip. Create an Eclipse launch configuration which defines the property to true and you have a convenient way to switch between the two.

If you want to run a slow test, select the method in Eclipse (or the whole class) and use "Run as JUnit Test" from the context menu. Since the property is false by default, the tests will be run.

Answer 2

No, you cannot skip tests if they are already running.

What I suggest you do is use Categories to separate your slow tests from the rest of your tests.

For example:

public interface SlowTests {
}

public class MyTest {

     @Test
     public void test1{
     }

     @Category(SlowTests.class)
     @Test
     public void test1001{
         // this is a slow test
     }
}

Create a test suite for the fast tests.

@RunWith(Categories.class)
@ExcludeCategory(SlowTests.class)
@SuiteClasses(MyTest.class)
public class FastTestSuite {

}

Now execute the FastTestSuite if you don't want to run the slow tests (e.g. test1001). Execute MyTest as normal if you want to run all the tests.

Answer 3

What you're asking for is to stop executing your code while it is in mid test. You can't stop executing a current test without having hooks in your code to allow it. Your best solution is to use Categories as others have suggested.

Basically, JUnit executes all of the @Before methods (including @Rules), then your @Test method, then the @After methods (again, including @Rules). Even assuming that JUnit had a mechanism for stopping execution of it's bits of the code (which it doesn't), most of the time is spent in your code. So to 'skip' a test which has already started requires you to modify your test code (and potentially the code that it's testing) in order that you can cleanly stop it. Cleanly stopping an executing thread is a question in itself [*].

So what are your options?

1. Run the tests in parallel, then you don't have to wait as long for the tests to finish. This may work, but parallelizing the tests may well be a lot of work.

2. Stop execution of the tests, and fix the one that's you're working on. Most IDEs have an option to kill the JVM in which the tests are running. This is definitely the easiest option.

3. Implement your own test runner, which runs the test in a separate thread. This test runner then either waits for the thread to finish executing, or checks a flag somewhere which would be a signal for it to stop. This sounds complicated, because you need t manage your threads but also to set the flag in a running jvm. Maybe creating a file somewhere? This runner would then fail the currently running test, and you could move on to the next. Please note that 'stopping' a test midway may leave stuff in an inconsistent state, or you may end up executing stuff in parallel.

There are parallel JUnit runners out there, and I don't think you're going to get much help from IDE developers (at least in the short term). Also, look at TestNG, which allows stuff to be run in parallel.

For using categories, one solution I use is to run the long running tests separately using maven surefire or similar, not through the IDE. This involves checking out the source code somewhere else on my machine and building there.

[*]: Java, how to stop threads, Regarding stopping of a thread

Answer 4

I think a more common solution is to have two test suites: one for the fast tests and another for the slow ones. This is typically the way you divide unit tests (fast) and integration tests (slow).

It's highly unlikely that you'll get modifications to JUnit or IntelliJ for something like this. Better to change the way you use them - it'll get you to an answer faster.

Answer 5

You can modify your thest and do something like

public void theTest(){
    if (System.getProperty("skipMyTest") == null){
         //execute the test
    }
}

and pass the environment variable if you want to skip the test