How to enable a global timeout for JUnit testcase runs?

Question

This question suggests to use the timeout parameter of the @Test annotation to have JUnit forcefully stop tests after that timeout period.

But we have like 5000 unit tests so far, and we want to establish a policy that asks developers to never release tests that need more than 10 seconds to complete. The policy would probably say "aim for < 10 seconds", but then we would like to ensure that any test is stopped after say 30 seconds. (the numbers are just examples, the idea is to define something that is "good enough" for most use cases, but that also makes sure things dont run "forever" )

Now I am wondering if there is a way to enable such behavior without turning into each test case and adding that annotation parameter.

The existing question doesn't help either: I am looking for one change to enable this, not a one change per test class solution. One central, global switch. Not one per file or method.

Answer 1

Although JUnit Jupiter (i.e., the programming and extension model introduced in JUnit 5) does not yet have built-in support for global timeouts, you can still implement global timeout support on your own.

The only catch is that a timeout extension cannot currently abort test execution preemptively. In other words, a timeout extension in JUnit Jupiter can currently only time the execution of tests and then throw an exception if the execution took too long (i.e., after waiting for the test to end, which may potentially never happen if the test hangs).

In any case, if you want to implement a non-preemptive global timeout extension for use with JUnit Jupiter, here's what you need to do.

1. Look at the TimingExtension example in the JUnit 5 User Guide for inspiration. You'll need code similar to that, but you'll want to throw an exception if the duration exceeds a configured timeout. How you configure your global timeout is up to you: hard code it, look up the value from a JVM system property, look up the value from a custom annotation etc.
2. Register your global timeout extension using Java's ServiceLoader mechanism. See Automatic Extension Registration for details.

Happy Testing!

Answer 2

Check out my JUnit 4 extension library (https://github.com/Nordstrom/JUnit-Foundation). Among the features provided by this library is the ability to define a global timeout value, which will be automatically applied to each test method that doesn't already define a longer timeout interval.

This library uses the Byte Buddy byte code generation library to install event hooks at strategic points in the test execution flow of JUnit 4. The global timeout is applied when JUnit has created a test class instance to run an "atomic" test.

To apply the global timeout, the library replaces the original @Test annotation with an object that implements the @Test interface. This approach utilizes all of JUnit's native timeout functionality, which provides pre-emptive termination of tests that run too long. The use of native timeout functionality eliminates the need for invasive implementation or special-case handling, and this functionality is activated without touching a single source file.

All of the updates needed to install and activate global timeout support are in the project file (POM / build.gradle) and optional properties file. The timeout interval can be overridden via System property, which enables adjustments to be made from the command line or programmatically. For scenarios where timeout failures are caused by transient conditions, you may want to pair the global timeout feature with the automatic retry feature.

Answer 3

What you're probably looking for is not implemented: https://github.com/junit-team/junit4/issues/140

Although, you can achieve the same results with simple inheritance.

Define an abstract parent class, like BaseIntegrationTest with the following @Rule field:

public abstract class BaseIntegrationTest extends RunListener {

    private static final int TEST_GLOBAL_TIMEOUT_VALUE = 10;

    @Rule
    protected Timeout globalTimeout = Timeout.seconds(TEST_GLOBAL_TIMEOUT_VALUE);

}

Then make it a parent for every test class within the scope. For example:

public class BaseEntityTest extends BaseIntegrationTest {

    @Before
    public void init() {
        // init
    }

    @Test
    public void twoPlusTwoTest() throws Exception {
        assert 2 + 2 == 4;        
    }
}

That's it.

Answer 4

Currently, maybe you cannot because Junit 5 removed the Rule and replace with the Extension. The above example does not work because the example code implements the AfterTestExecutionCallback that will be invoked after the test method is done, so the timeout is not useful.