Running a set of @Order ordered JUnit5 tests multiple times

Question

I wanted to know if there was a straightforward way of running a set of ordered tests multiple times with JUnit5, as opposed to running each test multiple times with the @RepeatedTest annotation.

For example, my tests:

   @Test
   @Order(1)
    public void myFirstTest()  {

      //code here

    }

   @Test
   @Order(2)
    public void myFirstTest()  {

      //code here

    }

   @Test
   @Order(3)
    public void myFirstTest()  {

      //code here

    }

I want them to run sequentially, 1-3 but repeated once the sequence has finished. Can this be done easily in JUnit 5 or is a @RunWith(Parameterized.class) still the easiest way, as described here

score 0 · Answer 1 · answered Jul 11 '20 at 09:36

@RunWith(Parameterized.class) won't work in JUnit 5. You could use your own launcher. Which might be overkill but it works.

The test class with tests that needs to be repeated:

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
@Disabled
public class OrderedExampleTest {

    @Test
    @Order(1)
    public void shouldRunFirst() {
        System.out.println("First!! ");
    }

    @Test
    @Order(2)
    public void shouldRunSecond() {
        System.out.println("Second!!");
    }

    @Test
    @Order(3)
    public void shouldRunThird() {
        System.out.println("Third!!");
    }
}

And then in a separate class a special test method that will re-run this whole class multiple times:

public class ExternalRunnerTest {

    @Test
    public void shouldRepeatTestClass() {
        LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
                .selectors(
                        selectPackage("org.igorski.repeated"),
                        selectClass(OrderedExampleTest.class)
                )
                .configurationParameter("junit.jupiter.conditions.deactivate", "org.junit.*DisabledCondition") 
                .build();

        Launcher launcher = LauncherFactory.create();

        int numberOfRepeats = 5;
        for(int i = 0; i < numberOfRepeats; i++) {
            launcher.execute(request);
        }
    }
}

The OrderedExampleTest is disabled. So it should not run on its own when you run all the tests together. But, the special repeat test will ignore the disabling condition and run the test.

In my case this prints:

First!! 
Second!!
Third!!
First!! 
Second!!
Third!!
First!! 
Second!!
Third!!
First!! 
Second!!
Third!!
First!! 
Second!!
Third!!

Running a set of @Order ordered JUnit5 tests multiple times

1 Answers1