maven - fail build when unit test takes too long

Question

I have in my project a lot of unit tests, written in JUnit and TestNG. The building process is based on maven with surefire plugin.

Is there any way/plugin for maven to fail the build when at least one unit test takes too many seconds? I know that there are some plugins to fail build in TeamCity, Jenkins but this is "too far".

In my project I want to have only fast tests to have unit testing process effective. I can improve my old tests but I need to protect for the future commitments

Answer 1

If all your tests extend some base class, I think you can stick @Rule in there which will apply to all the @Test in the child class.

e.g.

import org.junit.rules.Timeout;
public abstract class BaseTestConfiguration {
    @Rule public Timeout testTimeout = new Timeout(60000); // 60k ms = 1 minute
}

public class ThingTests extends BaseTestConfiguration {
    @Test
    public void testThis() {
        /*will fail if not done in 1 minute*/
    }
    @Test
    public void testThat() {
        /*will fail if not done in 1 minute*/
    }
}

We found this to be easiest rather than messing with AspectJ or secret configuration directives. Devs are more likely to figure out the Java parts than all the secret XML this and that. You can put @Before @After and any number of other junit directives in the base class too.

Answer 2

In maven surefire, you can use forkedProcessTimeoutInSeconds, along with forkMode=once.

This will kill the forked jvm if it takes too long. If you want to do this per test, you can forkMode=pertest or forkMode=always (which does it for each class).

Answer 3

Why don't add a timeout on a per test basis. I assume you're programming in Java, so in JUnit you can do something like this:

 @Test(timeout=100) public void testQuickly()

If the test doesn't end after 100 milliseconds, it will fail.

Answer 4

Have you tried specifying the timeout for the test through @Test(timeout=xx)? Find the official api documentation here: http://junit.sourceforge.net/javadoc/org/junit/Test.html

EDIT---

You can also consider using the timeout property for the TestNg suite, you will have to move all your tests to TestNg though, but this will allow you to specify timeouts for groups.

This is the official api documentation: http://testng.org/javadoc/org/testng/xml/XmlSuite.html#setTimeOut(java.lang.String)

Answer 5

Try to use the junit Ant task with the timeout parameter.

http://ant.apache.org/manual/Tasks/junit.html

EDIT:
Another thing you can do is to use aspectj with someting like that:

public aspect TestTimeoutChecker {
    long TIMEOUT = 60000;

    pointcut invokeEvent() :
        execution(@Test * *(..));

    Object around() : invokeEvent() {

        long start = System.currentTimeMillis();
        Object object = proceed();
        long took = System.currentTimeMillis() - start;
        if (took > TIMEOUT) {
            throw new RuntimeException("timeout! it took:" + took);
        }
        return object;
    }
}

Answer 6

I once had the same requirement, and since I was using TestNG, I used the "group feature".

If you're stuck with JUnit, try using the tests suite (http://stackoverflow.com/questions/817135/grouping-junit-tests)