Test default value and setter in same test-case or separate test cases

Question

Would you recommend doing any grouping of test cases within @Test methods, or have one @Test method per test scenario? For example, let's suppose that there are different ways to set the context in an application.

Is the following idea acceptable?

@Test
public void testContextSetting() {
    // Test default setting
    assert(...)

    // Test setting a context variable
    assert(...)

    ...
}

Or, would you rather suggest having it like this, having each method as atomic as possible:

@Test
public void textDefaultSetting() {
    // Test default setting
    assert(...)
}

@Test
public void testSettingContextVar() {
    // Test setting a context variable
    assert(...)

    ...
}

Any feedback would be appreciated.

This is still a useful read: http://www.exubero.com/junit/antipatterns.html (see Multiple Assertions section) — planetjones, Jun 06 '11 at 12:54
See also [Unit Testing: One Test, One Assertion – Why It Works.](http://blog.astrumfutura.com/2009/02/unit-testing-one-test-one-assertion-why-it-works/) — avandeursen, Mar 02 '12 at 20:51

score 35 · Accepted Answer · edited Jun 20 '14 at 13:24

I prefer having one test case per method.

First it is easier to see what cases are being tested if they are split into methods as opposed to looking for comments embedded in the code. Most IDEs will give you a summary of methods, so instead of saying "did I test edgecase XYZ?" and then hunting for a comment, or looking for the code that sets up that edgecase, you just look for the method named setupContextEdgeCaseXYZ().

A second reason is if you have multiple cases together one may fail and then the others never execute.

 testDefaultCase()
 testInvalidInput()
 testEdgeCase1()
 testEdgeCase2()

With this structure it would be easier to determine that the input checking is bad and edge case 2 is handled improperly, but the others are OK (and you may find out that two failing cases are related and the problem is diagnosed faster).

A third reason is you may accidentally leave values from a previous test set that invalidates a latter test in a inconspicuous way. A simple example:

@Test
public void testMyMethod() {
  //test default
  String test = Foo.bar(null);
  assertEquals("foo", test);

  //test case 1
  Foo.bar(aValue);
  //Oops forgot to set value above, this passes regardless of 
  //what the above call does
  assertEquals("foo", test);
}

By breaking cases apart you can avoid mistakes as above as that would turn into a compile error or warning.

+1 But I would suggest a case in which it is better to have a single test: when your test fixture is hard/costly to setup. — Felipe Kamakura, Aug 28 '13 at 16:39
And of course if it is a fixture that can be shared among various tests.In that case a single test would save you from setting up the fixture multiple times. Often that happens in integration tests where one uses in-memory databases, Spring, that sort of things. — Felipe Kamakura, Aug 28 '13 at 16:46

score 11 · Answer 2 · answered Jun 06 '11 at 12:46

11

Best Practice is to have one test case per method. The method name describes the test that you are performing. It is easier to debug when your test fails when it is just one assert.

answered Jun 06 '11 at 12:46

RMT

7,040
4
25
37

3

i think "Best Practice is to have one test case per method" should be "Best Practice is to have one assert per test case". – Martin Dürrmeier Sep 22 '14 at 10:20

score 11 · Answer 3 · answered Jun 06 '11 at 12:53

11

Divide et impera :) so split in multiple small cases ...easier to fix in case of errors.

answered Jun 06 '11 at 12:53

Cris

4,947
6
44
73

score 1 · Answer 4 · answered Dec 09 '14 at 13:41

You are confusing a test with an assertion. Your test method with multiple asserts tests multiple things: default setting and setting a context variable. But a test method that tests one thing can also have multiple asserts.

A good pattern to use is for each test-case to have four phases:

Setup: where you create the objects you need to perform the test, and if necessary alter those objects to put them in the required initial states.
Exercise: where you perform the operation that you are testing. This will be one method call, or a constructor call.
Verify: where you check that the objects under test are in the correct state(s), and check the value returned by the method you called in the exercise phase, if it returned a value. This is where you place your asserts. If you use this pattern, there is nothing wrong with placing multiple asserts in the verify phase.
Teardown: where you destroy or close the objects you used to perform the test.

That is the approach recommended in the book xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros.

Contrast that pattern with what you have in your first example, in which it seems you would do this:

Initial setup
Exercise constructor
Verify default
Exercise to set a context variable
Verify setting of context variable
Teardown

score 0 · Answer 5 · answered Jun 11 '15 at 11:01

Eclipse generates unit test per method which seems to be a reasonable approach. If the tested method is too complex to test it using one test method then You may consider re factoring it.

But a much more better approach is to use TDD and write the tests upfront which will drive the rest of the design and implementation.

Personally I prefer to have one test per method along with the Java Code Coverage for Eclipse http://www.eclemma.org.

The tool will tell You what You're actually testing.

Test default value and setter in same test-case or separate test cases

5 Answers5

Linked