Best practices for unit testing in Python - multiple functions apply to same object

Question

I have a bunch of functions which apply to a similar object, for example a Numpy array which represents an n-dimensional box:

# 3-D box parameterized as:
#     box[0] = 3-D min coordinate
#     box[1] = 3-D max coordinate
box = np.array([
    [1, 3, 0],
    [4, 5, 7]
])

Now I have a whole bunch of functions that I want to run on lists of boxes, eg. volumes, intersection, smallest_containing_box, etc. In my mind here is the way I was hoping to set this up:

# list of test functions:
test_funcs = [volume, intersection, smallest_containing_box, ...]
# manually create a bunch of inputs and outputs
test_set_1 = (
    input = [boxA, boxB, ...], # where each of these are np.Array objects
    output = [
        [volA, volB, ...], # floats I calculated manually
        intersection, # np.Array representing the correct intersection
        smallest_containing_box, # etc.
    ]
)
# Create a bunch of these, eg. test_set_2, test_set_3, etc. and bundle them in a list:
test_sets = [test_set_1, ...]
# Now run the set of tests over each of these:
test_results = [[assertEqual(test(t.input), t.output) for test in test_funcs] for t in test_sets]

The reason I want to structure it this way is so that I can create multiple sets of (input, answer) pairs and just run all the tests over each. Unless I'm missing something, the structure of unittest doesn't seem to work well with this approach. Instead, it seems like it wants me to create an individual TestCase object for each pair of function and input, i.e.

class TestCase1(unittest.TestCase):
    def setUp(self):
        self.input = [...]
        self.volume = [volA, volB, ...]
        self.intersection = ...
        # etc.

    def test_volume(self):
        self.assertEqual(volume(self.input), self.volume)

    def test_intersection(self):
        self.assertEqual(intersection(self.input), self.output)

    # etc.

# Repeat this for every test case!?

This seems like a crazy amount of boilerplate. Am I missing something?

[This answer](https://stackoverflow.com/a/34094/1060781) may be what I was looking for. I looks like I could write one `TestCase`, make a tuple of inputs and outputs, and then use the `subTest` context to run each test function over all the inputs and outputs. If this is the "right approach" then mods can feel free to close this as a duplicate, but if there is something better / more idiomatic I would be very interested to hear about it. — mboratko, Mar 17 '19 at 15:36
In addition, if I'm just thinking about unit testing in the wrong way (i.e. there is some significant benefit from splitting it up some other way) I would like to know that as well :) — mboratko, Mar 17 '19 at 15:48

score 0 · Answer 1 · answered Mar 20 '19 at 16:50

Let met try to describe how I understand your approach: You have implemented a number of different functions that have a similarity, namely that they operate on the same types of input data. In your tests you try to make use of that similarity: You create some input data and pass that input data to all of your functions.

This test-data centric approach is unusual. The typical unit-testing approach is code-centric. The reason is, that one primary goal of unit-testing is to find bugs in the code. Different functions have (obviously) different code, and therefore the types of bugs may be different. Thus, test data is typically carefully designed to identify certain kinds of bugs in the respective code. Test design methods are approaches that methodically design test cases such that ideally all likely bugs would be detected.

I am sceptical that with your test-data centric approach you will be equally successful in finding the bugs in your different functions: For the volume function there may be overflow scenarios (and also underflow scenarios) that don't apply for intersection or smallest_containing_box. In contrast, there will have to be empty intersections, one-point-intersections etc.. Thus it seems that each of the functions probably needs specifically designed test scenarios.

Regarding the boilerplate code that seems to be the consequence of code-centric unit-testing: There are several ways to limit that. You would, agreed, have different test methods for different functions under test. But then you could use parameterized tests to avoid further code duplication. And, for the case that you still see an advantage to use (at least sometimes) common test-data for the different functions: For such cases you can use factory functions that create the test-data and can be called from the different test cases. For example, you could have a factory function make-unit-cube to be used from different tests.

score 0 · Answer 2 · answered Mar 20 '19 at 17:07

Try unittest.TestSuite(). That gives you an object where you can add test cases. In your case, create the suite, then loop over your lists, creating instances of TestCase which all have only a single test method. Pass the test data to the constructor and save them to properties there instead of in setUp().

The unit test runner will detect the suite when you create it in a method called suite() and run all of them.

Note: Assign a name to each TestCase instance or finding out which failed will be very hard.

Best practices for unit testing in Python - multiple functions apply to same object

2 Answers2