20

JUnit4 has @FixMethodOrder annotation which allows to use alphabetical order of test methods execution. Is there analogous JUnit5 mechanism?

Radek Postołowicz
  • 4,506
  • 2
  • 30
  • 47

3 Answers3

23

Edit: JUnit 5.4 is officially released now, so no need to use snapshots anymore.

This is now possible with JUnit 5.4.

https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-execution-order

To control the order in which test methods are executed, annotate your test class or test interface with @TestMethodOrder and specify the desired MethodOrderer implementation. You can implement your own custom MethodOrderer or use one of the following built-in MethodOrderer implementations.

Alphanumeric: sorts test methods alphanumerically based on their names and formal parameter lists.

OrderAnnotation: sorts test methods numerically based on values specified via the @Order annotation.

Alexander Abramov
  • 1,470
  • 14
  • 20
  • It works for me, yet adding only `dependencies { testImplementation "org.junit.jupiter:junit-jupiter-api:5.4.0-SNAPSHOT" testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:5.4.0-SNAPSHOT" }` is enough – Alexander Vasiljev Nov 30 '18 at 05:59
  • I use maven pom.xml. At this moment I can see 5.4.0-M1 version. As I know Milestone Dependacies not good for developments. How do you guys get a different version? :O – Menuka Ishan Jan 03 '19 at 11:31
  • How can I order test classes? – dmatej May 13 '19 at 10:34
  • 1
    @dmatej Currently you cannot. But you can add a comment if you need it as well: https://github.com/junit-team/junit5/issues/1948 – JiangHongTiao Mar 10 '20 at 15:19
13

No, not yet. For unit tests, execution order should be irrelevant. For more complex tests, JUnit is aiming to provide explicit support - test ordering would be part of that.

Nicolai Parlog
  • 47,972
  • 24
  • 125
  • 255
  • 3
    From a technical perspective, it's true that execution order should be irrelevant, but from a human perspective, the number of times that the random order has tripped me up is too much. I want them to execute in the same order from test run to test run. – Adam Jul 07 '17 at 09:14
  • 3
    I'd argue that if the non-deterministic order trips you up, then the code you're testing has implicit temporal dependencies that make it very hard to use. I'd make fixing that a high priority, which has the added benefit to stabilize the test suite. – Nicolai Parlog Jul 08 '17 at 07:32
  • 4
    Disagree, from the UI perspective. The human brain. E.g. I'm running a class of ten tests and 2 are failing, so my immediate training tells me to look at the two tests that are failing, in order from the top, like the 2nd one down and the 4th one down, when I run it again. But then when they don't come back in the same order, I have to (a) realise that there's not some bizarre non-deterministic bug causing random tests to fail, they're just not in the same order and (b) remember the usually dumb test names that failed and run it again to make sure they are actually failing consistently. – Adam Jul 08 '17 at 12:31
  • So technically I totally agree with you - running in random order is good - but for the sake of sanity, I'd like a little button on the IDE unit test dialog which allows me to impose order on the test run - preferably by order of appearance in code. – Adam Jul 08 '17 at 12:31
  • Ah, I misunderstood what you meant by getting "tripped up" - thought you were talking about coding. Re UX, what you describe is indeed suboptimal but I don't have that problem. Can't speak for other IDEs but IntelliJ allows to list tests alphabetically and to rerun only failed tests. – Nicolai Parlog Jul 10 '17 at 05:55
  • 1
    Yes I use Intellij at the moment, and I like the re-run failed tests feature - but the 'run-in-code-order' is my ideal and I end up naming the test methods so their code order is also alphabetical. – Adam Jul 10 '17 at 08:16
  • That's actually not a bad idea and might be worth [a feature request](https://youtrack.jetbrains.com/issues/IDEA). – Nicolai Parlog Jul 10 '17 at 18:43
0

With version 5.8.0 onwards, test classes can be ordered too.

src/test/resources/junit-platform.properties:

# ClassOrderer$OrderAnnotation sorts classes based on their @Order annotation
junit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer$OrderAnnotation

Other Junit built-in class orderer implementations:

org.junit.jupiter.api.ClassOrderer$ClassName
org.junit.jupiter.api.ClassOrderer$DisplayName
org.junit.jupiter.api.ClassOrderer$Random

For other ways (beside junit-platform.properties file) to set configuration parameters refer here.

You can also provide your own orderer. It must implement ClassOrderer interface:

package foo;
public class MyOrderer implements ClassOrderer {
    @Override
    public void orderClasses(ClassOrdererContext context) {
        Collections.shuffle(context.getClassDescriptors());
    }
}

junit.jupiter.testclass.order.default=foo.MyOrderer

Note that @Nested test classes cannot be ordered by a ClassOrderer.

Refer to JUnit 5 documentations and ClassOrderer api docs to learn more about ordering test classes.

Mahozad
  • 18,032
  • 13
  • 118
  • 133