How to make JUnit test cases to run in sequential order?

Question

I am using JUnit4.

I have a set of test methods in a test case.

Each test method inserts some records and verify a test result and finally delete the records inserted.

Since the JUnit run in parallel, test methods fail because of some records present during the execution of previous test method. This happen only in my colleague machine(Windows 7), not in my machine(Cent OS 6).

What we need is that the test methods have to pass in all our machines.

I have tried clearing the records in the Setup() method but again it works only on my machine. Is there any option available in JUnit to make the test methods to run in a uniform sequential order ?

Thanks,

score 31 · Answer 1 · edited Nov 29 '17 at 17:38

31

JUnit 4.11 now supports specifying execution order using @FixMethodOrder annotation.

edited Nov 29 '17 at 17:38

mernst

7,437
30
45

answered Dec 17 '12 at 10:13

Aleksandr Dubinsky

22,436
15
82
99

score 31 · Answer 2 · edited Jun 24 '16 at 08:52

MethodSorters is a new class introduced after Junit 4.6 release. This class declared three types of execution order, which can be used in your test cases while executing them.

NAME_ASCENDING(MethodSorters.NAME_ASCENDING) - Sorts the test methods by the method name, in lexicographic order.
JVM(null) - Leaves the test methods in the order returned by the JVM. Note that the order from the JVM my vary from run to run.
DEFAULT(MethodSorter.DEFAULT) - Sorts the test methods in a deterministic, but not predictable, order.

.

import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;

//Running test cases in order of method names in ascending order

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class OrderedTestCasesExecution {

    @Test
    public void secondTest() {
        System.out.println("Executing second test");
    }

    @Test
    public void firstTest() {
        System.out.println("Executing first test");
    }

    @Test
    public void thirdTest() {
        System.out.println("Executing third test");
    }
}

Output:

Executing first test
Executing second test
Executing third test

Reference: http://howtodoinjava.com/2012/11/24/ordered-testcases-execution-in-junit-4/

Now (in 2020), I found another feature to assign order of methods, I don't know when this feature introduced. It uses @Order(a positive number here) to assign ordinal. https://www.baeldung.com/junit-5-test-order#:~:text=1.-,Overview,%2C%20but%20unpredictable%20order%20(MethodSorters. — Daniel Yang, Mar 08 '21 at 09:45
@DanielYang That is for JUnit 5, but the question is for JUnit 4. — Leponzo, Mar 10 '21 at 16:10

score 10 · Answer 3 · answered Mar 12 '12 at 13:36

Ordering of tests is not guaranteed in JUnit.

The reason for this is that unit tests are meant to be atomic - all of the setup should happen in the setup / tear down methods, but not by other tests.

Consider moving the code that inserts data into another helper class that can be called by both the test that's inserting and the class that needs to verify, and calling that class in your @Before methods.

You should also consider a mocking solution (eg Mockito) as opposed to hitting the database directly if you can - mocking will go a long way to ensuring that your tests are nice and isolated, and, as a nice side benefit, usually help point out where you could use some refactoring.

You could also use something like DBUnit to control your database setup & teardown, but for pure unit testing Mockito makes more sense as it allows tests to be completely stand-alone. — TrueDub, Mar 14 '12 at 08:39

score 3 · Answer 4 · edited May 23 '17 at 11:54

Because you're running the tests in parallel, and you're hitting the database, you're highly likely to have problems, because the database won't necessarily be in a coherent state for each test.

Solution: don't run your tests in parallel. JUnit doesn't run the tests in parallel by default, so either you're setting the option in maven or using one of the parallel runners in JUnit.

If you're still having problems between tests failing on Windows but not on Cent OS, then it's maybe a problem with run order, which you'll need to fix. See my answer to Has JUnit4 begun supporting ordering of test? Is it intentional?.

The way around this (at least in JUnit terms) is to remove the dependencies between tests. Basically, JUnit doesn't support ordering and the tests should be able to be run in any order.

If you really need to have dependencies between tests, use TestNG, where you can have dependencies.

score -2 · Answer 5 · answered Jun 23 '15 at 02:55

There is no problem running tests in parallel even if you have your data layer in it. But you need to have additional work to create MOCK UPs for your data so not it will not hit the database. You can use different mockup frameworks like Mockito, EasyMock and Arquillian.

How to make JUnit test cases to run in sequential order?

5 Answers5

Linked

Related