8

I have a testcase that looks like this:

def MyTestCase(unittest.Testcase):
  def test_input01(self):
    input = read_from_disk('input01')
    output = run(input)
    validated_output = read_from_disk('output01')
    self.assertEquals(output, validated_output)
  def test_input02(self):
    input = read_from_disk('input02')
    # ...
  # and so on, for 30 inputs, from input01 to input30

Now, I understand that test code can be a bit repetitive, since simplicity is more important than conciseness. But this is becoming really error-prone, since when I decided to change the signature of some functions used here, I had to make the change in all 30 places.

I could refactor this into a loop over the known inputs, but I do want each input to remain a separate test, so I thought I should be making the test_inputxx methods.

What am I doing wrong?

max
  • 49,282
  • 56
  • 208
  • 355
  • do the inputs need to be separate test cases? Why not just have a list of inputs and loop through them? – monkut Sep 11 '12 at 02:43
  • Each input represents a different scenario. And I'd like to see which of them fail, and have such information collected and reported all at once, for these and all my other tests. I thought that's the whole reason for using unittest framework to begin with? – max Sep 11 '12 at 02:44

4 Answers4

13

Write a helper function to remove the repetition from the test cases:

def MyTestCase(unittest.Testcase):
  def run_input_output(self, suffix):
    input = read_from_disk('input'+suffix)
    output = run(input)
    validated_output = read_from_disk('output'+suffix)
    self.assertEquals(output, validated_output)

  def test_input01(self): self.run_input_output('01')
  def test_input02(self): self.run_input_output('02')
  def test_input03(self): self.run_input_output('03')
Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • Ahh.. I don't know why I didn't try that. I `unittest` scares me with its complexity, so I was afraid to add any method other than `setUp` or `test_xxx`... – max Sep 11 '12 at 03:01
  • Do I understand correctly that `setUp` / `tearDown` should not be used in this example (even if technically I could somehow push into them the name of the test to be run or that just ran)? – max Sep 11 '12 at 09:21
  • @max: I don't understand why setUp or tearDown wouldn't be involved? The code here is just three tests that run like any other. The fact that a helper method is involved doesn't change that. – Ned Batchelder Sep 11 '12 at 11:51
2

I like Ned Batchelder's solution. But for posterity, if you might often change the number of inputs, you could do something like:

def MyTestCase(unittest.Testcase):
    def __init__(self, *args, **kwargs):
        for i in range(1,31):
            def test(self, suffix=i):
                input = read_from_disk('input%02d' % suffix)
                output = run(input)
                validated_output = read_from_disk('output%02d' % suffix)
                self.assertEquals(output, validated_output)
            setattr(self, 'test_input%02d' % i) = test
        super(MyTestCase, self).__init__(*args, **kwargs)
jcater
  • 501
  • 4
  • 7
  • This is a lot more concise, which I personally like. But should I be prepared to be chased by furious colleagues with farming utensils for writing non-Pythonic / non-obvious code, in a testing module of all places? – max Sep 11 '12 at 03:05
  • 1
    Yes, prepare to be chased. Test code should have no trickery. – Jon Reid Sep 11 '12 at 03:53
  • 3
    Indeed. Wear your best running shoes. – jcater Sep 11 '12 at 04:02
  • Would any important test tools (especially `nose`) miss these tests because they are added too late? – max Sep 11 '12 at 11:08
  • And in fact, even unittest won't pick these up. They need to be defined through metaclasses. nose won't pick these up even with metaclasses. The working solution seems to be this: http://stackoverflow.com/a/676420/336527. – max Sep 11 '12 at 11:16
1

How about something like this, so that it reports which input failed.

def MyTestCase(unittest.Testcase):
  def test_input01(self):
    for i in range(1,30):
      input = read_from_disk('input%.2d' %i)
      output = run(input)
      validated_output = read_from_disk('output%.2d' %i)
      self.assertEquals(output, validated_output, 'failed on test case %.2d' %i)
yass
  • 41
  • 1
  • 5
  • 1
    But if test #3 raises an exception, the whole loop will break, and other tests won't finish, will they? I could put `output = run(input)` inside try/except. And manually implement other features I need (like `setUp`, `tearDown`). But then why use unittest at all? – max Sep 11 '12 at 02:51
  • I believe you want `%.2d` instead of `%2d` so that you get `01` instead of `1`. Also it looks like you're writing in ruby instead of python? You don't exactly need those ends. Solid code though. – Nolen Royalty Sep 11 '12 at 02:51
  • @NolenRoyalty, thanks! I corrected it based on your suggestions. – yass Sep 11 '12 at 02:54
  • For the record I think that using `%.2d` like that is a really cool solution, I never would have thought of that :p. +1 now that you've fixed it. – Nolen Royalty Sep 11 '12 at 02:56
1

My favorite tool for this kind of test is parameterized test cases, which look like this:

from nose_parameterized import parameterized

class MyTestCase(unittest.TestCase):
    @parameterized.expand([(1,), (2,), (3,)])
    def test_read_from_disk(self, file_number):
        input = read_from_disk('input%02d' % file_number)
        expected = read_from_disk('output%02d' % file_number)

        actual = run(input)
        self.assertEquals(expected, actual)

You write your test case to take whatever parameters you need, wrap the parameterized function in the @parameterized.expand decorator, and provide sets of input parameters inside the expand() call. The test runner then kindly runs an individual test for each set of parameters!

In this case, there's only one parameter, so the expand() call has an unfortunate extra level of nestedness, but the pattern becomes especially nice when your use case is a little more complex and you use param objects to provide args and kwargs to your test function:

from nose_parameterized import parameterized, param

class MyTestCase(unittest.TestCase):
    @parameterized.expand([
        param(english='father', spanish='padre'),
        param(english='taco', spanish='taco'),
        ('earth', 'tierra'), # A regular tuple still works too, but is less readable
        ...
    ])
    def test_translate_to_spanish(self, english, spanish):
        self.assertEqual(translator(english), spanish)

The pattern allows you to easily and clearly specify many sets of input parameters, and only write the testing logic once.

I use nose for testing, so my example uses nose-parameterized, but there's also a unittest-compatible version.

waterproof
  • 4,943
  • 5
  • 30
  • 28