Python Unittests continue after failure

Question

What's the best way of continuing tests after failure in unittest?

#!/usr/env/bin python2.7

import unittest
from fib import fib


class FibTests(unittest.TestCase):

    def test_0(self):
        self.assertEqual(fib(0), 0)
        self.assertEqual(fib(1), 1)
        self.assertEqual(fib(2), 1)
        self.assertEqual(fib(5), 5)
        self.assertEqual(fib(10), 55)

    def test_1(self):
        self.assertEqual(fib(0), 1)

    def test_2(self):
        self.assertEqual(fib(1), 0)
        self.assertEqual(fib(5), 0)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(FibTests)
    result = unittest.TextTestRunner(verbosity=2).run(suite)

Looking at test_2, it'll only tell me there's 1 failure instead of 2.

Additionally, how can I collect the results at the end stating:

test_0 -- 0 failures
test_1 -- 1 failures
test_2 -- 2 failures

Motivation:

I'm trying to create a testing game. People submit tests and if they fail others' programs they get points; each test failure is one point. What's the easiest way of providing this type of capability?

Things whose failures you want to keep track of separately should be separate tests. — BrenBarn, Jan 17 '13 at 04:41
each assert will raise a fail. If you want each to be their own testcase you need to make them their own testcase method. — monkut, Jan 17 '13 at 04:41

score 1 · Accepted Answer · answered Jan 17 '13 at 07:34

You have to separate each test case into different method. To make it easier to create new test / save typing time, you can auto-generate test case based on input like this

#!/usr/env/bin python2.7

import unittest
from fib import fib


class FibTests(unittest.TestCase):
    pass

def test_gen(expected, actual):
    def test_method(self):
        return self.assertEqual(expected, actual)
    return test_method

if __name__ == '__main__':
    cases = ((fib(1), 1), (fib(2), 1), (fib(5), 5))
    for index, case in enumerate(cases):
        test_name = 'test_{0}'.format(index)
        test = test_gen(case[1], case[0])
        setattr(FibTests, test_name, test)
    suite = unittest.TestLoader().loadTestsFromTestCase(FibTests)
    result = unittest.TextTestRunner(verbosity=2).run(suite)

You could do something messy like this, or you could just use `nose`, which supports generator test functions. — Silas Ray, Jan 18 '13 at 02:00

Python Unittests continue after failure

1 Answers1

Linked