3

I'm a bit new to Python's unittest library and I'm currently looking at setting up my Flask server before running any of my integration tests. I know that the unittest.TestCase class allows you to use setUp() before every test cases in the class. I also know that the same class has another method called setUpClass() that runs only once for the entire class.

What I'm actually interested is trying to figure out how to do something similar like setUpClass(), but done on an entire unittest.TestSuite. However, I'm having no luck at it.

Sure, I could set up the server for every TestCase, but I would like to avoid doing this.

There is an answer on a separate question that suggests that by overriding unittest.TestResult's startTestRun(), you could have a set up function that covers the entire test suite. However, I've tried to passed in the custom TestResult object into unittest. TextTestRunner with no success.

So, how exactly can I do a set up for an entire test suite?

Joseph Woolf
  • 500
  • 5
  • 14

1 Answers1

3

This is not well documented, but I recently needed to do this as well.

The docs mention that TestResult.startTestRun is "Called once before any tests are executed."

As you can see, in the implementation, the method doesn't do anything.

I tried subclassing TestResult and all kinds of things. I couldn't make any of that work, so I ended up monkey patching the class.

In the __init__.py of my test package, I did the following:

import unittest


OLD_TEST_RUN = unittest.result.TestResult.startTestRun

def startTestRun(self):
    # whatever custom code you want to run exactly once before 
    # any of your tests runs goes here:
    ...

    # just in case future versions do something in this method
    # we'll call the existing method
    OLD_TEST_RUN(self)

unittest.result.TestResult.startTestRun = startTestRun

There is also a stopTestRun method if you need to run cleanup code after all tests have run.

Note that this does not make a separate version of TestResult. The existing one is used by the unittest module as usual. The only thing we've done is surgically graft on our custom implementation of startTestRun

Stephen
  • 2,613
  • 1
  • 24
  • 42
  • Thank you for the reply. So I've put this code inside of ```__init__.py```, but it never runs unless I use a TestSuite. Why is this method very buggy/nonfunctional? – Joseph Woolf Mar 01 '21 at 04:40
  • @JosephWoolf How are you running your unit tests? – Stephen Mar 04 '21 at 02:21
  • So, the first approach that I used was to use a TestSuite and import the modified unittest fron ```__init__.py```. While this did work, the issue was that the classes that held the tests ran twice, which isn't what I want. – Joseph Woolf Mar 04 '21 at 02:43
  • The second approach was going into the file containing the test cases can importing the modified unittest from ```__init__.py```. However, this approach didn't work at all – Joseph Woolf Mar 04 '21 at 02:43
  • We're also using nose2 to run our tests – Joseph Woolf Mar 04 '21 at 02:44
  • If you want to use the solution in this answer run your unit tests with the standard library. e.g.: `python -m unittest`. This is the reason running your tests with nose2 doesn't work: https://docs.nose2.io/en/latest/differences.html#internals – Stephen Mar 04 '21 at 16:18
  • You might be able to leverage a nose2 feature if you need to use nose2 instead of `unittest` from the std lib. Perhaps this or something on this page will help: https://docs.nose2.io/en/latest/dev/hook_reference.html#startTestRun – Stephen Mar 04 '21 at 16:19
  • @JosephWoolf to be clear: the only thing in the `__init__.py` should be what I have in the answer. No need to import your tests. Run your test suite via `python -m unittest` – Stephen Mar 05 '21 at 19:55
  • @JosephWoolf it's also worth mentioning that if you are just testing your flask app, there isn't a need to run your flask server, just use the test client: https://flask.palletsprojects.com/en/1.1.x/api/#flask.Flask.test_client – Stephen Mar 05 '21 at 19:59