How to unittest multiple similar classes

Question

I have a program with multiple very similar classes:

class BlackBox1():
    def calc(self, a, b):
        return a + b

class BlackBox2():
    def calc(self, a, b):
        return a * b
...

Now I want to write unittests for all those classes. Of course I could write separate tests for each Blackbox. Anyway, since each blackbox has exactly the same method calc(a, b) to be tested, I wonder, if there is something like a “best practise”, to automatically give classes and expected results to a abstract test framework, something like

import unittest
class TestAbstractBox(unittest.TestCase):
    def setUp(self):
        self.box = blackbox()
        self.param_a = a
        self.param_b = b
        self.expected_result = result

    def test_calc_method(self):
        real_result = self.box.calc(self.param_a, self.param_b)
        self.assertEqual(real_result, self.expected_result, 
                        "{0} gives wrong result".format(self.box.__class__))

TAbstractTest = unittest.defaultTestLoader.loadTestsFromTestCase(TestAbstractBox)

Is there a way to pass {"blackbox": Blackbox1, "a": 3, "b": 5, "result": 8} and {"blackbox": Blackbox2, "a": 4, "b": 7, "result": 28} to the TestAbstractBox class to not have multiple times the same code, but have a easy way to test new Blackboxes?

freakish · Answer 1 · 2013-04-05T06:59:44.957

1

You can simply add these classes to setUp:

class TestAbstractBox(unittest.TestCase):
  def setUp(self):
    self.boxes = [Blackbox1(), Blackbox2()]
    self.param_a = a
    self.param_b = b
    self.expected_result = result

  def test_calc_method(self):
    for box in self.boxes:
      real_result = self.box.calc(self.param_a, self.param_b)
      self.assertEqual(real_result, self.expected_result, 
          "{0} gives wrong result".format(self.box.__class__))

EDIT Version 2:

class TestAbstractBox(unittest.TestCase):
  def setUp(self):
    self.boxes = [
      { "blackbox":Blackbox1(), "a":a1, "b":b1, "expected_result":result1 },
      { "blackbox":Blackbox2(), "a":a2, "b":b2, "expected_result":result2 },
    ]

  def test_calc_method(self):
    for box in self.boxes:
      real_result = box["blackbox"].calc(box["a"], box["b"])
      self.assertEqual(real_result, box["expected_result"], 
          "{0} gives wrong result".format(box["blackbox"].__class__))

edited Apr 05 '13 at 06:59

answered Apr 05 '13 at 06:11

freakish

54,167
9
132
169

But using this way, I would need to have arrays for the parameters and results as well, and since I have quite a lot of blackboxes, this would soon become confusing. – Sebastian Werk Apr 05 '13 at 06:13
@SebastianWerk Well you can keep one array of dictionaries of the format you've shown us: `{"blackbox": Blackbox1, "a": 3, "b": 5, "result": 8}`. You still have to pass these params at some point. – freakish Apr 05 '13 at 06:16
@freakish I suck at unittesting and I'm doing a lot of katas to get better, is this really the best approach for a unittest? Also, if he wants to have more than 1 test wouldn't that allow for the setUp method to have both Blackboxes attached to `self` even though he just wants to test something else like a `Blackbox()` being `None` for example? Since the `setUp()` runs for each test, right? Btw, this is a question not critique! :D – Henrik Andersson Apr 05 '13 at 06:19
@limelights He can use other variables for other tests. To make it compact, he could just store params for this test in one variable and loop over it (array of dicts). It is a simple solution, does not require neither tricks nor creating addditional classes (see: OP's answer). – freakish Apr 05 '13 at 06:55
@freakish could you join here? http://chat.stackoverflow.com/rooms/27614/unittest-conceptual – Henrik Andersson Apr 05 '13 at 07:11
@limelights Hey, sorry, had to go to work. I'm going to be on stackoverflow for most of the day, so let me know when you're available (and please wait around 15minutes). – freakish Apr 05 '13 at 08:13

score 0 · Accepted Answer · edited May 23 '17 at 10:25

I think I found a solution, basically following this answer:

import unittest

class BlackBox1():
    def calc(self, a, b):
        return a + b

class BlackBox2():
    def calc(self, a, b):
        return a * b

class TestBlackBox1(unittest.TestCase):
    test_params = {"blackbox": BlackBox1, "a": 3, "b": 5, "result": 8}
    def setUp(self):
        self.box = self.test_params["blackbox"]()
        self.param_a = self.test_params["a"]
        self.param_b = self.test_params["b"]
        self.expected_result = self.test_params["result"]

    def test_calc_method(self):
        real_result = self.box.calc(self.param_a, self.param_b)
        self.assertEqual(real_result, self.expected_result,
                        "{0} gives wrong result: {1} instead of {2}".format
                        (self.box.__class__, real_result, self.expected_result))

class TestBlackBox2(TestBlackBox1):
    test_params = {"blackbox": BlackBox2, "a": 4, "b": 7, "result": 28}

TBB1 = unittest.defaultTestLoader.loadTestsFromTestCase(TestBlackBox1)
TBB2 = unittest.defaultTestLoader.loadTestsFromTestCase(TestBlackBox2)

Anyway, thanks for the other answers and sorry for not finding this way in the first place.

And how is that better then just simply keeping an array of dicts in `setUp`? It only makes your test more complicated. — freakish, Apr 05 '13 at 06:57
Just to name the most important ones: If additional tests for some blackboxes are needed, they can be easily added to the class, if I have more than one test, your answer would require each time a for loop, I see only the first failing test in test_calc_method (so if multiple boxes are broken, I only get them step by step). Since this is a framework, were other people can add further blackboxes, I can give them an easy way to test their boxes. I also like your example, but for my case, this works better and appears more objective oriented. — Sebastian Werk, Apr 05 '13 at 07:04

score 0 · Answer 3 · answered Apr 05 '13 at 07:08

I don't think this is such a good approach.

A test method should do exactly one test, not loop over several assert* calls. Otherwise in case of a failure you would not be able to to easily tell which test failed. After all that's the whole point of doing unittests.

Also, you should not risk breaking existing tests by adding new variants for new classes, or refactoring existing tests into some common methods.

In the case of several classes with identical methods (think of implementing a common interface in several derived classes) it may be useful to put all the tests for the common properties of the interface in to one base class and then deriving individual test classes for each class under test. Those derived classes would then implement suitable setUp and tearDown methods. They may also add more test cases specific to the class under test.

How to unittest multiple similar classes

3 Answers3