53

In this post, I asked a small question as part of a bigger problem. Since I didn't get responses yet, I put my question here:

Is it reasonable to suppose that JUnit executes test cases sequentially: a test case ends befores the next one starts. Does it differ between JUnit versions (my priority is on JUnit4)? And if not, is there a simple way to force JUnit to execute tests sequentially?

Thank you

Community
  • 1
  • 1
H-H
  • 4,431
  • 6
  • 33
  • 41
  • As a side note, here is some more about executing tests in parallel using JUnit 4.7: http://stackoverflow.com/questions/423627/running-junit-tests-in-parallel – mort Sep 01 '11 at 08:39
  • If you use JUnit within your Maven build, and your build is set to be multi-threaded using the -T parameter, (e.g. -T 4C), then the tests will be ran in parallel where possible – Tom Chamberlain Aug 18 '15 at 10:43

7 Answers7

36

Yes, by default I believe it runs tests sequentially.

JUnit 4.6 introduced an experimental ParallelRunner to run tests in parallel - I don't know of its current status. I would expect parallelism to remain an "opt-in" feature by default though. (Judging by other answers, it looks like this is now here to stay but in a slightly different form - and still opt-in.)

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Is it generally better to run tests in parallel instead of sequentially ? What kind of hardware must we have in order to run tests in parallel ? – MasterJoe Mar 09 '18 at 03:35
  • Does this sound like an acceptable way of confirming sequence other than debugging ? Create a test class with 10 test methods. Create a list of integers in test class. Each test method inputs its position in class into the List. Run all test 10 times. Each time, the integers must always be in correct order. – MasterJoe Mar 09 '18 at 03:51
  • 1
    @testerjoe2: I wouldn't rely on that, to be honest. I'd trust the documentation more than just ad-hoc testing - it would be very easy for there to be some heuristics at play (maybe "sequential for 10 methods or fewer, parallel for more" for example) which would give a false positive. – Jon Skeet Mar 09 '18 at 06:24
21

Parallel execution of tests is supported since JUnit 4.7. But as far as I know, it is never done automatically, you specifically need to configure it, e.g. like here: http://java.dzone.com/articles/running-junit-tests-parallel

But don't forget that:

Good automated tests should be independent, isolated and reproducible, making them ideal candidates for being run concurrently.

I don't know why you ask, but if the above criteria is not met, you might want to think about your test design.

mort
  • 12,988
  • 14
  • 52
  • 97
  • 1
    "Good automated tests should be independent, isolated and reproducible" - Can we achieve this by having separate test data for each test, as long as the test data creation process is not too time/resource consuming ? – MasterJoe Mar 09 '18 at 03:33
  • 2
    Can you please provide the source of this quote? – wotopul Aug 20 '18 at 22:03
  • The source of the quote seems to be a DZone article from 2010: https://dzone.com/articles/running-junit-tests-parallel – Ortomala Lokni Nov 30 '20 at 17:18
6

A strong and reasonable guess: yes, JUnit is single threaded by default.

Otherwise one wouldn't know if a test failed because the code is broken or it failed because of concurrency problems if some tests ran in parallel.

Rudy Velthuis
  • 28,387
  • 5
  • 46
  • 94
Andreas Dolk
  • 113,398
  • 19
  • 180
  • 268
  • If each test uses its own test data (not always possible), then is it better to run tests in parallel ? – MasterJoe Mar 09 '18 at 03:37
  • If you are able to write thread safe code and data-independant tests, then yes. We use some custom config that spins up multiple independent test dabases when running tests, so that we can have parallell runners that don't mix. Make sure you don't have hidden shared state, like `static`s. – oligofren Sep 17 '21 at 12:05
6

Yes,

And also think of the @before and @after there you may have code that restores the state for the next test to run.

4

By default, JUnit runs tests using a deterministic, but unpredictable order (MethodSorters.DEFAULT).

In Junit 5: if you want to run your tests sequentially

  • use @TestMethodOrder to control the execution order of tests.
  • And then use the @Order annotation to enforce tests to run in a specific order.
  • Example:

@TestMethodOrder(OrderAnnotation.class)
public class OrderAnnotationUnitTest {
    private static StringBuilder output = new StringBuilder("");

    @Test
    @Order(1)
    public void firstTest() {
        output.append("a");
    }

    @Test
    @Order(2)
    public void secondTest() {
        output.append("b");
    }

    @Test
    @Order(3)
    public void thirdTest() {
        output.append("c");
    }

    @AfterAll
    public static void assertOutput() {
        assertEquals(output.toString(), "abc");
    }
}
Taslim Oseni
  • 6,086
  • 10
  • 44
  • 69
Apoorv Gupta
  • 35
  • 3
1

Yes. As mentioned somewhere in the comments, you should plan carefully the testcase setup and teardown (all the superclasses affect those actions), as well as testsuite setup and teardown.

Also, on a secondary note, as far as I can remember, JUnit does not guarantee the order of execution of the testcases (that is, unless they are in a suite, I guess). This is important, and should push you to performing a very precise cleanup and SUT state restoration between tests, and avoid test cases relying on the results of other testcases. you might say that this is a kind of antipattern :).

Adam
  • 561
  • 1
  • 4
  • 6
-8

With my experience i can say we can serialize tests in a class by naming the method name having Test in it. ex:

@Test

public ..... firstTest ()
{

}

@Test

public .... thirdTest()
{

}

@Test

public....secondTest()
{

}

The order is firstTest,thirdTest,secondTest

Andrew Barber
  • 39,603
  • 20
  • 94
  • 123
sri
  • 1