Stop testsuite if a testcase find an error

Question

I have a testSuite in Python with several testCases.

If a testCase fails, testSuite continues with the next testCase. I would like to be able to stop testSuite when a testCase fails or be able to decide if the testSuite should continue or stop.

As your unit tests should be independent, this requirement seems odd. Why not execute all unit tests, so you see all failing tests at once? — Daniel Hilgarth, Jul 25 '11 at 09:27

score 16 · Answer 1 · answered Jul 26 '11 at 10:35

16

Since Python 2.7, unittest support failfast option. It can be either specified by commandline:

python -m unittest -f test_module

Or when using a script:

>>> from unittest import main
>>> main(module='test_module', failfast=True)

Unfortunately I haven't yet found out how to specify this option when you are using setuptools and setup.py.

answered Jul 26 '11 at 10:35

plaes

31,788
11
91
89

Note: this doesn't work if you want to use `unittest discover` (or I just put the flag in the wrong place?). – szeitlin Aug 04 '21 at 16:20

score 3 · Answer 2 · answered Jan 28 '20 at 19:36

3

None of the answers seem to touch on this part of your question:

... be able to decide if the testSuite should continue or stop.

You can override the run() method within your TestCase class and call the TestResult.stop() method to signal to the TestSuite to stop running tests.

class MyTestCase(unittest.TestCase):
    def run(self, result=None):
        if stop_test_condition():
            result.stop()
            return
        super().run(result=result)

answered Jan 28 '20 at 19:36

raspberrybi

41
4

Good answer to the actual question. I was able to bracket my call to super().run() with a handler for a custom exception raised by setUp(). (For [hacky](https://stackoverflow.com/a/45518359/673991) reasons, the exception has to be a subclass of KeyboardInterrupt.) – Bob Stein Nov 20 '20 at 14:39

score 2 · Answer 3 · answered Nov 07 '19 at 15:18

2

Use failfast=True it will stop running all tests if 1 fails in your test class

Example:

if __name__ == '__main__':
    unittest.main(failfast=True)

answered Nov 07 '19 at 15:18

Jet_C

644
9
12

score 1 · Answer 4 · answered Jul 25 '11 at 09:18

1

Are you actually doing unit tests? Or system tests of something else? If the latter, you may be interested in my Python based testing framework. One of its features is exactly this. You can define test case dependencies and the suite will skip tests with failed dependencies. It also has built-in support to selenium and webdriver. However, it's not so easy to set up. Currently in development but mostly works. Runs on Linux.

answered Jul 25 '11 at 09:18

Keith

42,110
11
57
76

Thanks Keith but I don't know how to use your Python based testing framework to solve my problem. – Gabriel Quesada Jul 25 '11 at 09:34

score 1 · Answer 5 · answered Jul 25 '11 at 09:35

Run your tests with nose and use the -x flag. That combined with the --failed flag should give you all you need. So in the top level of your project run

nosetests -x # with -v for verbose and -s to no capture stdout

Alternatively you could run with

nosetests --failed

Which will re-run only your failing tests from the test suite

Other useful flags:

nosetests --pdb-failure --pdb

drops you into a debugger at the point your test failed or errorred

nosetests --with-coverage --cover-package=<your package name> --cover-html

gives you a colorized html page showing which lines on your code have been touched by the test run

A combination of all of those usually gives me what I want.

I'm not sure - I know unittest2 brings a lot of new goodies, but I haven't really used them — Ben Ford, Jul 25 '11 at 10:39

score -3 · Answer 6 · answered Jul 25 '11 at 09:12

-3

You can use sys.exit() to close the python interpreter at any point within your testcase.

answered Jul 25 '11 at 09:12

Senthil Kumaran

54,681
14
94
131

I want to close the python interpreter within my testSUITE when running the testSUITE an error happens – Gabriel Quesada Jul 25 '11 at 09:53
1

I don't think this is true, `unittest` appeared to catch the `sys.exit` and continue when I tried this solution – qwwqwwq Feb 04 '14 at 02:20
-1: neither `sys.exit()` nor `sys._exit()` actually halt my tests. – sarnold May 08 '14 at 22:30

Stop testsuite if a testcase find an error

6 Answers6

Linked