How to set custom test execution order for a test suite in JUnit5?

Question

I have a large set of tests on JUnit5, which I run in parallel in several threads. There is also information about the time of each test. I want to run at the beginning of the longest tests, and leave the fastest at the end to optimize common execution time.

I have not found a way to do this in JUnit5.

In version 5.4 there is an org.junit.jupiter.api.MethodOrderer interface which allows you to write a test sorter within a test class. And connect to the test class via the annotation org.junit.jupiter.api.TestMethodOrder.

I would like something similar, but globally, for the whole test suite.

Answer 1

It is now possible to order test classes in JUnit 5 (since v5.8.0).

src/test/resources/junit-platform.properties:

# ClassOrderer$OrderAnnotation sorts classes based on their @Order annotation
junit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer$OrderAnnotation

Other Junit built-in class orderer implementations:

org.junit.jupiter.api.ClassOrderer$ClassName
org.junit.jupiter.api.ClassOrderer$DisplayName
org.junit.jupiter.api.ClassOrderer$Random

For other ways (beside junit-platform.properties file) to set configuration parameters refer here.

You can also provide your own orderer. It must implement ClassOrderer interface:

package foo;
public class MyOrderer implements ClassOrderer {
    @Override
    public void orderClasses(ClassOrdererContext context) {
        Collections.shuffle(context.getClassDescriptors());
    }
}

junit.jupiter.testclass.order.default=foo.MyOrderer

Note that @Nested test classes cannot be ordered by a ClassOrderer.

Refer to JUnit 5 documentations and ClassOrderer api docs to learn more about ordering test classes.

Answer 2

What you're asking for is unfortunately not currently possible in JUnit 5 (i.e., either on the JUnit Platform or within JUnit Jupiter).

There is however an open issue dedicated to this topic. So feel free to chime in there.