Determine the most practical number of testcases for a method

Question

Assume you have a method check() of a class in your Business Layer that does some validations on instances of this class. The checks are of the form if attribute a of that class has a certain value, attribute b needs to have a certain other value (that is a determines b). The method could for instance internally use a map containing all valid combinations of a and b:

public bool check() {
   if (mapOfValidCombos.get(a)!=b)
        return false;
   return true;
}

or it could look them up in an external csv file or it could consist of a list of if statements:

public bool check() {
    if (a==1) and (b!=5) then return false;
    if (a==2) and (b!=3) then return false;
    return true;
}

Now if you would like to unit test that method. What are the testcases you should implement:

You could test for all valid combinations of a and b, and also the invalid ones (assuming a and b are enumerable types)
You could only create one testcase per path of that method (e.g. one for a valid combination, one for an invalid one)

Approach number 2 result in less unit testcases (and hence less maintenance effort for them). However where is the right place to actually test that the method is doing the right checks from a business perspective (e.g. if the map of valid combinations has the correct values in it?).

If you go for approach number 1 should you add additional testcases if you change the implementation from lets say Map bases to if statements since you get more executtion paths of your method?

EDIT: I was thinking more in terms of what value these tests would add to our test suite. Let's say if go for a map based implementation of the check() method what use is there to duplicate the content of the map into my test case. Regarding hidro hint to use paraterized tests. They still need to get their parameters from somewhere (and most likely not out of check()'s implementation) so the values are somehow duplicated. I was thinking more along the line that the test of the actual values and the completeness of the check method is something done by QA or a business departement in acceptance testing, where as my unit test just check the execution paths of my code and not necessarily all the combinations of allowed values. Any thoughts?

A boss of mine once asked what the minimum amount of testing we could do. My answer: "none". — Raedwald, Feb 28 '15 at 11:41
Just to share here, not to argue. At my company, we have at least 2 levels of testing. Unit testing is where engineers unit test code, make sure unit tests cover all execution paths (they are they only one who control those execution paths anyway), and after that acceptance tests by QA, where the metrics are to cover as many business features as possible (which may not directly map to execution paths, as some are very rare to happen). — hidro, Feb 28 '15 at 11:44
@Raedwald: :-) nice one, I am looking for the right ammount of testing though... — markus, Feb 28 '15 at 15:58
@hidro that sounds reasonable to me, so a consequence of that is if I'd refactor my code to reduce execution path's I could could also reduced the number of tests I'd maintain. In the rare case that my refactoring leads to more execution path I'd have to add more testcases. — markus, Feb 28 '15 at 16:00

score 0 · Accepted Answer · answered Feb 27 '15 at 08:06

My rules of thumb are:

Try to have only one assert per test method if possible. Only use multiple asserts if they are meant to be evaluated together.
Write tests with single assert until you cover all execution paths, which means several tests may share the same assert of output, but with different input values that all lead to that same expectation.

Now if you start to see lots of duplication in your test class, no sweat.

'Parameterize' your test. I'm not sure which language you are using, but many of them support parameterized test, e.g. PHPUnit's @dataProvider or Java's JunitParams, which is given a 'table' of test specs, execute the same test code (same assert).
Avoid writing utility methods or classes to reuse test code, it's more anti-pattern than helpful in the long run.

We are using Cpp/CppUnit. It does not support parametrization out of the box but there are possibilities to extend it to get it done, see: http://stackoverflow.com/questions/290099/how-to-parametrize-a-test-using-cppunit and for an example implementation of parametrizable Macros http://brain-child.de/engineering/parameterizing-cppunit-tests/ see my edit above — markus, Feb 28 '15 at 11:30

Determine the most practical number of testcases for a method

1 Answers1