2

I would like to test an API, which received one argument and returns a set. The test invokes the API with an argument and checks if the returned set contains expected values.

Suppose, I have to test the API with arguments arg1, arg2, and arg3 and check if values a, b, c appear in the returned set. That is, my test case looks as follows:

  • invoke the API with arg1 and check if a, b, c appear in the returned set.
  • invoke the API with arg2 and check if a, b, c appear in the returned set.
  • invoke the API with arg3 and check if a, b, c appear in the returned set.

How to develop this test case with Junit 4 ? What if I have to add arg4 ? What if I have to check if value d appear in the returned set ? Can I read the list of arguments and expected values from the configuration?

Michael
  • 41,026
  • 70
  • 193
  • 341

4 Answers4

4

Fluent assertions

First of all, use FEST-Assertions library to introduce pleasantly looking assertions with meaningful error messages:

assertThat(method(arg1)).containsExactly(a, b, c);
assertThat(method(arg2)).containsExactly(a, b, c);
assertThat(method(arg3)).containsExactly(a, b, c);

The BDD way

But I understand your question is not about the syntax, but about methodology: what should you do if arg4 needs to be tested? Well, if arg1 through arg4 have a different semantic meaning, I would advice you to have a separate test for each argument. Very verbose, but also extremely readable (pseudocode):

@Test
public void shouldReturnAbcWhenSomeArgumentUsed() {
  //given
  Object arg = arg1;

  //when
  Set<Object> result = method(arg);

  //then
  assertThat(result).containsExactly(a, b, c);
}

..and repeat this for every test. The key part is: method name should represent the meaning of an argument, what does this method actually test, what do you expect, what is the scenario?

Consider testing isEven method. I would recommend the following tests:

  • shouldReturnTrueForZero
  • shouldReturnTrueForSmallPositiveEvenNumber
  • shouldReturnTrueForLargePositiveEvenNumber
  • shouldReturnFalseForSmallPositiveOddNumber
  • shouldReturnFalseForLargePositiveOddNumber
  • ... and mirror the tests for negative numbers

Each test represent a slightly different, well defined scenario. On the other hand you might generate literally thousands of shouldReturnFalseWhen227, but what is the value of such a test suite, except it's large? Test semantically different arguments and corner cases, defining precisesly what case is being tested.

Parameterized way

If you really want to have just a single generic test method, Parameterized runner is the way to go. I think the example is self-explanatory. NB: you can parameterize expected values as well.

@RunWith(value = Parameterized.class)
public class JunitTest6 {

    private Object arg;

    public JunitTest6(Object arg) {
        this.arg = arg;
    }

    @Parameterized.Parameters
    public static Collection<Object[]> data() {
        return Arrays.asList(
                new Object[][]{
                        {arg1},
                        {arg2},
                        {arg3}
                });
    }

    @Test
    public void testMethod() {
        assertThat(method(arg)).containsExcatly(a, b, c);
    }

}

Based on this.

Tomasz Nurkiewicz
  • 334,321
  • 69
  • 703
  • 674
  • Thanks. You get it right. I am asking about the methodology. My problem is that I would not like to write different test methods for all those arguments. – Michael Jun 05 '11 at 14:02
  • 1
    @Michel Testing methodology is a vast subject and it depends of many things, in particular what the thing you test is supposed to do and how it do it. To be pratical you should try to do as little testing as possible while checking for edge cases and ensuring that each part of your code as been tested at least once. The theory says that you can't test everything anyway, that you don't have time, and that your test will have bugs too. Even the question you asked is very specific to a simple pure functions as many thing doesn't always return the same thing if called several time. – Nicolas Bousquet Jun 05 '11 at 14:32
  • Thoses example you showed are nice and all Tomasz, but excepted from the first, they are really verbose. The first solution is only 3 line long but the second solution is 33 line long (3 methods to write), the last is 25 lines long. One core principe in software engineering is KISS (Keep it simple). I understand that Michell want to see nice methodologic exemple, but used without case thoses methodology might very well do more harm than good. – Nicolas Bousquet Jun 05 '11 at 14:46
3

I would usually turn to Hamcrest for something like this -- it's a library for declaratively writing "matchers", and it plays very nicely with JUnit.

However, this question on SO points out that although this can be done with Hamcrest, a simpler way is just to use the containsAll method from java.util.Collection:

ArrayList<Integer> expected = new ArrayList<Integer>();
expected.add(1); expected.add(2); expected.add(3);

assertTrue(actual.containsAll(expected));
Community
  • 1
  • 1
Todd Owen
  • 15,650
  • 7
  • 54
  • 52
3

In terms of methodology:

The "agile" way

Tests need ongoing development and refactoring, just like production code. As well, principles like YAGNI ("you ain't gonna need it") apply too. If right now you only need to test a, b and c then I would start with an ordinary hard-coded unit test. If later your test cases start becoming repetitive then by all means consider how to refactor them.

Or maybe you are already at that point now, but to me the question doesn't seem to provide enough information to give a more specific suggestion about how to refactor the unit tests. Read tests from XML? Generate combinatorial test data? Parameterized runner (as per @Tomasz)? Maybe I just haven't understood the question well enough, but the problem as stated seems too abstract still.

Todd Owen
  • 15,650
  • 7
  • 54
  • 52
1

assuming you want to test the method "method1" that take one parameter and return a set you would write:

Set result = method1(arg1);
assertTrue(result.contains(a));
assertTrue(result.contains(b));
assertTrue(result.contains(c));

But maybe the best would be to compare the set directly with it expected value:

Set expected = new HashSet();
expected.add(a);
expected.add(b);
expected.add(c);

assertEquals(expected, method1(arg1));
assertEquals(expected, method1(arg2));
assertEquals(expected, method1(arg3));

And of course don't hesiste to use loops to be more generic if needed.

Nicolas Bousquet
  • 3,990
  • 1
  • 16
  • 18
  • Thanks but I would like to have something more generic and dynamic. That's why I am asking what if I add more arguments and expected values to check. – Michael Jun 05 '11 at 13:58
  • This won't work at all. In the way it's used here, the assertEquals() calls will compare object references, not their contents. They will always fail, even if the method works as expected. – Bohemian Jun 05 '11 at 14:06
  • More values to check ? just put more elements to your "expected" Set. If this become tedious, you can load argument and expected result from a file. JUnit just bring some commodities (asserts, tests methods, reporting...) the rest is just normal code. You can write the same generic code you'll write for other part of your software. Beware through that being too generic will make the test difficult to understand and maintain. As a generic assert failing in a loop without any helpfull message will not help a lot. – Nicolas Bousquet Jun 05 '11 at 14:11
  • @Bohemian, the assertEquals method call the underlying equals method that should be defined to whatever is relevant for type equality. In particular JAVA collections redefine equal to something more usefull than comparing references. It is assertSame that compare references. – Nicolas Bousquet Jun 05 '11 at 14:19