What's the difference between / advantage of using JUnit 5's @ParametrizedTest over @TestFactory Stream?

Question

First, what do they mean by this in the conclusion of Guide to Dynamic Tests in Junit 5?

The parameterized tests can replace many of the examples in this article. However, the dynamic tests differ from the parameterized tests as they support full test lifecycle, while parametrized tests don’t.

I skimmed through JUnit 5 – Parameterized Tests and believe I understand the differences at a syntactic level, and also believe I get this:

Moreover, dynamic tests provide more flexibility regarding how the input is generated and how the tests are executed.

But then it seems like, why would anyone prefer parametrized tests over dynamic tests?

My guess is: parameterized tests are with a strict parameter set, whereas dynamic tests use random data? — Lino, Feb 28 '19 at 09:37
Hm, actually from what I read it seemed both could do both, _i.e._ use a strict parameter set like an array, or generate inputs and outputs. @ParametrizedTest seemed to use something called "method sources" (and "custom argument sources"), while for DynamicTest you had to pass in a lambda generator no matter what. But besides, I feel like the bit about "full test lifecycle" was pointing to something else... — Andrew Cheong, Feb 28 '19 at 09:38

Sormuras · Accepted Answer · 2019-02-28T09:59:29.193

Dynamic tests, I refer to them as testlets, are just soft/grouped-assertions (assertAll(...)) on steroids. You'll see an entry for each generated dynamic test in the reports but they are not real tests.

The quote (from baeldung) your copied into your question is wrong. It should read as in JUnit's User-Guide:

Dynamic Test Lifecycle

The execution lifecycle of a dynamic test is quite different than it is for a standard @Test case. Specifically, there are no lifecycle callbacks for individual dynamic tests. This means that @BeforeEach and @AfterEach methods and their corresponding extension callbacks are executed for the @TestFactory method but not for each dynamic test.

For more details read: https://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests

Why would anyone prefer parametrized tests over dynamic tests?

If you need full lifecyle support for each test (template invocation).
If you want to declare your arguments in annotations.

Find a lot more details on how to provide arguments in various forms to a @ParameterizedTest here: https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests -- mind that "class/container templates" are slated for a later release: https://github.com/junit-team/junit5/issues/878

I wrote a blog post comparing 5 steps of scattering 3 assertions with JUnit Jupiter here: https://sormuras.github.io/blog/2018-05-14-junit5-scatter-assertions.html

Amazingly helpful, thank you. Would like to see if anyone else has more to say before I accept. — Andrew Cheong, Feb 28 '19 at 10:05

Ken Chan · Answer 2 · 2019-02-28T10:37:40.300

More flexibility somehow means it is more complex and boilerplate to write especially most of the time the test case is rather static but not so dynamic.

Consider I want to test Math.add() :

The parameterized test version looks likes :

@ParameterizedTest
@CsvSource({ "1,1,2",
            "2,2,4",
            "3,3,6",
            "4,4,8",
            "5,5,10",
            "6,6,12",
            "7,7,14",
            "10,90,100" })
public void parameterizedTest(int left, int right, int expected) {
    assertEquals(expected, Math.addExact(left, right));
}

The dynamic test version looks likes:

@TestFactory
Collection<DynamicTest> dynamicTest() {
    return Arrays.asList(
      DynamicTest.dynamicTest("Test1", () -> assertEquals(2, Math.addExact(1, 1))),
      DynamicTest.dynamicTest("Test2", () -> assertEquals(4, Math.addExact(2, 2))),
      DynamicTest.dynamicTest("Test3", () -> assertEquals(6, Math.addExact(3, 3))),
      DynamicTest.dynamicTest("Test4", () -> assertEquals(8, Math.addExact(4, 4))),
      DynamicTest.dynamicTest("Test5", () -> assertEquals(10, Math.addExact(5, 5))),
      DynamicTest.dynamicTest("Test6", () -> assertEquals(12, Math.addExact(6, 6))),
      DynamicTest.dynamicTest("Test7", () -> assertEquals(14, Math.addExact(7, 7))),
      DynamicTest.dynamicTest("Test8", () -> assertEquals(100, Math.addExact(10, 90))));
}

It already has many boilerplate codes.So I try to use return Stream<DynamicTest> to remove these boilerplate codes :

@TestFactory
Stream<DynamicTest> dynamicTest2() {

    return Stream.of(
                "1,1,2", 
                "2,2,4", 
                "3,3,6",
                "4,4,8" ,
                "5,5,10" , 
                "6,6,12" ,
                "7,7,14", 
                "10,90,100")
                //How to do????????
            .map(data-> DynamicTest.dynamicTest(data, () -> assertEquals(xxx, Math.addExact(yy,zz))));
}

But how can I convert my test data which is in the string format to the arguments and call the SUT. I look around DynamicTest API to see if there are anything can help me but can't find anything helpful, so I give up .....

So ,I would prefer parametrised test. More elegant ,clean, easy to read and write. Readability of the test case is more important.

Hm, but in this case couldn't you use a stream from `int[][]`, _i.e._ `{{1, 1, 2}, {2, 2, 4} ... }` for the dynamic tests? I do agree readability is important but for me @CsvSource doesn't seem compelling enough a reason... — Andrew Cheong, Feb 28 '19 at 10:41
Yes , you are right . Although it is not a rocket science , I have difficulties to create and access multidimensional array in Java ,especially need to map and access from the correct index of the stream , not to mentioned 4 , 5 , 6 dimension array cases...... :P — Ken Chan, Feb 28 '19 at 10:51
Ah, well I know how that feels too haha. Maybe I'm just looking for one more reason other than the convenience of a helper like @CsvSource. Thanks for your answer, still! — Andrew Cheong, Feb 28 '19 at 10:52

What's the difference between / advantage of using JUnit 5's @ParametrizedTest over @TestFactory Stream?

2 Answers2