What's the best way to set up a per-test or per-class timeout when using in perBatch forkmode?

Question

I want to fail the build if anyone writes a test that takes longer than 1 second to run, but if I run in perTest mode it takes much, much longer.

I could probably write a custom task that parses the junit reports and fails the build based on that, but I was wondering if anyone knows or can think of a better option.

score 33 · Accepted Answer · edited Oct 15 '21 at 16:42

Reviving an old question since the answer doesn't provide an example.

You can specify timeout

Per test method:

 @Test(timeout = 100) // Exception: test timed out after 100 milliseconds
 public void test1() throws Exception {
     Thread.sleep(200);
 }

For all methods in a test class using Timeout @Rule:

 @Rule
 public Timeout timeout = new Timeout(100);

 @Test // Exception: test timed out after 100 milliseconds
 public void methodTimeout() throws Exception {
     Thread.sleep(200);
 }

 @Test
 public void methodInTime() throws Exception {
     Thread.sleep(50);
 }

Globally for the total time to run all test methods in the class using a static Timeout @ClassRule:

 @ClassRule
 public static Timeout classTimeout = new Timeout(200);

 @Test
 public void test1() throws Exception {
     Thread.sleep(150);
 }

 @Test // InterruptedException: sleep interrupted
 public void test2() throws Exception {
     Thread.sleep(100);
 }

And even apply timeout (either @Rule or @ClassRule) to all classes in your entire suite:

 @RunWith(Suite.class)
 @SuiteClasses({ Test1.class, Test2.class})
 public class SuiteWithTimeout {
     @ClassRule
     public static Timeout classTimeout = new Timeout(1000);

     @Rule
     public Timeout timeout = new Timeout(100);
 }

EDIT: timeout was deprecated recently to utilize this initialization

@Rule
public Timeout timeout = new Timeout(120000, TimeUnit.MILLISECONDS);

You should provide the Timeunit now, as this will provide more granularity to your code.

Nice. I stopped using JUnit quite some time back. The last option looks good, although what I was looking for at the time was a way to timeout all tests run from Ant/Gradle globally so as to enforce, say, a 1 second timeout for every test in the project (essentially as a constraint for everyone working in this 400-person project to contend with). — Jun-Dai Bates-Kobashigawa, Aug 16 '15 at 22:45
If I use @Rule with espresso and Robolectric the tests fail with 'Can't create handler inside thread that has not called Looper.prepare()' — David Berry, Aug 19 '19 at 19:46
To use this in Kotlin, see this answer: https://stackoverflow.com/questions/29945087/kotlin-and-new-activitytestrule-the-rule-must-be-public — frodo2975, Feb 08 '23 at 19:39

score 8 · Answer 2 · answered Jan 05 '12 at 14:27

8

If you use JUnit 4 and @Test, you can specifiy the timeout parameter that will fail a tests that's taking longer than specified. Downside for that is that you'd have to add it to every test method.

A probably better alternative is the use of a @Rule with org.junit.rules.Timeout. With this, you can do it per class (or even in a shared super class).

answered Jan 05 '12 at 14:27

ftr

2,105
16
29

The Timeout `@Rule` is nice. Pity you can't do this on a `@Suite`. Or can you? – avandeursen Jan 06 '12 at 15:39
Hmm. Rather than adding @Rule to every class, I wonder if there's a relatively easy way to add it to all the classes at runtime? – Jun-Dai Bates-Kobashigawa Jan 10 '12 at 22:58
@Jun-DaiBates-Kobashigawa - You can use a custom test case base class as you used to do with Junit 3. Not ideal but a decent workaround. – David Harkness Oct 29 '12 at 22:38

score 2 · Answer 3 · answered Oct 30 '22 at 02:02

I know from the tags that this question was aimed at JUnit 4, but in JUnit 5 (Jupiter), the mechanism has changed.

From the guide:

   @Test
   @Timeout(value = 500, unit = TimeUnit.MILLISECONDS)
   void failsIfExecutionTimeExceeds500Milliseconds() {
       // fails if execution time exceeds 500 milliseconds
   }

(The unit defaults to seconds, so you could just use @Timeout(1).)

It's easy to apply once across a class:

To apply the same timeout to all test methods within a test class and all of its @Nested classes, you can declare the @Timeout annotation at the class level.

For your purposes, you might even experiment with top-level configuration on your build jobs:

"junit.jupiter.execution.timeout.test.method.default"
Default timeout for @Test methods

What's the best way to set up a per-test or per-class timeout when using in perBatch forkmode?

3 Answers3

Linked