Bootstrapping tests and using Python test discovery

Question

A problem I continue to have it "bootstrapping" my tests.

The problem that I have is exactly what this guy has.

The top solution talks about creating a "boostrap" script. I presume that I must then enumerate all of the tests to be run, or use test manifests in the __init__.py files using the __all__ keyword. However, I noticed that the most recent Python documentation on unittest does not talk about __all__ anymore.

In 2.7, we have the python command called "discovery"

python -m unittest discover

That works even nicer. Because: 1) There's no need for Nose 2) There's no need for test manifests

But it doesn't seem to have a way to "bootstrap"

Do I need to use another test runner? One that allows bootstrapping AND discovery?

Do I need py.test?
http://pytest.org/

The reason that I need bootstrapping, is the problem that this guy has. Basically, my import statements don't work right if I run the test directly. I want to execute my suite of tests from the top of my project, just like the app would when it runs normally.

After all, import statements are always relative to their physical location. (BTW, I think this is a hindrance in Python)

Definition: What is Bootstrapping? Bootstrapping means that I want to do some setup before running any tests at all in the entire project. This is sort of like me asking for a "test setup" at the whole project level.

Update Here is another posting about the same thing. Using this 2.7 command, we can avoid Nose. But how does one add bootstrapping?

So the problem is that you want to some way to do discovery + bootstrapping when you're not at the top of your project? That seems like a very limited use case, I only ever run tests from the top of my projects. — djc, Sep 15 '11 at 14:23
I think I may have mis-typed. I do want to start it from the top of my project -- just as though I were running my app normally. — 101010, Sep 16 '11 at 00:34

score 4 · Accepted Answer · answered Sep 16 '11 at 01:29

I got it!

Using this one script that I wrote and called it "runtests.py" and placed in my project root, I was able to "bootstrap" that is to run some initialization code AND use discovery. Woot!

In my case, the "bootstrap" code is the two lines that say:

import sys
sys.path.insert(0, 'lib.zip')

Thanks!

#!/usr/bin/python

import unittest
import sys
sys.path.insert(0, 'lib.zip')

if __name__ == "__main__":
    all_tests = unittest.TestLoader().discover('.')
    unittest.TextTestRunner().run(all_tests)

score 0 · Answer 2 · answered Sep 15 '11 at 15:29

Here's what I do, and I think it works quite well. For a file/directory structure similar to this:

main_code.py
run_tests.py
    /Modules
        __init__.py
        some_module1.py
        some_module2.py
    /Tests
        __init__.py
        test_module1.py
        test_module2.py

It's fairly easy to organize your run_tests.py file to bootstrap the tests. First every file with test (test_module1.py, etc.) should implement a function that generates a test suite. Something like:

def suite():
    suite = unittest.TestSuite()
    suite.addTest(unittest.makeSuite(Test_Length))
    suite.addTest(unittest.makeSuite(Test_Sum))
    return suite

at the end of your test code. Then, in the run_tests.py file, you aggregate these into an additional test_suite, and run that:

import unittest
import Tests.test_module1 as test_module1
import Tests.test_module2 as test_module2

module1_test_suite = test_module1.suite()
module2_test_suite = test_module2.suite()

aggregate_suite = unittest.TestSuite()
aggregate_suite.addTest(module1_test_suite)
aggregate_suite.addTest(module2_test_suite)

unittest.TextTestsRunner(verbosity = 2).run(aggregate_suite

Then to run all of these tests, from the command line, simply run

python run_tests.py

Your answer describes a variation of the "create a manifest" using the "all" keyword. I'm trying to avoid all of this work. I presume that the new "discovery" feature in Python 2.7 is meant for us to avoid all this extra work. — 101010, Sep 16 '11 at 00:36
That's fair. I guess I didn't understand that your problem occurred because you wanted to be able to run each test separately. Just for reference, I do this by passing flags as command line arguments, and then pulling them out of `sys.argv`, so that I can selectively import tests. This allows some fancier behavior like "don't run any integration tests". — Wilduck, Sep 16 '11 at 13:55

Bootstrapping tests and using Python test discovery

2 Answers2