How to organize python test in a way that I can run all tests in a single command?

Question

Currently my code is organized in the following tree structure:

src/
    module1.py
    module2.py
    test_module1.py
    test_module2.py
    subpackage1/
        __init__.py
        moduleA.py
        moduleB.py
        test_moduleA.py
        test_moduleB.py

Where the module*.py files contains the source code and the test_module*.py contains the TestCases for the relevant module.

With the following comands I can run the tests contained in a single file, for example:

$ cd src
$ nosetests test_filesystem.py
..................
----------------------------------------------------------------------
Ran 18 tests in 0.390s

OK

How can I run all tests? I tried with nosetests -m 'test_.*' but it doesn't work.

$cd src
$ nosetests -m 'test_.*'

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

Thanks

score 16 · Answer 1 · answered Dec 14 '08 at 18:41

Whether you seperate or mix tests and modules is probably a matter of taste, although I would strongly advocate for keeping them apart (setup reasons, code stats etc).

When you're using nosetests, make sure that all directories with tests are real packages:

src/
    module1.py
    module2.py
    subpackage1/
        __init__.py
        moduleA.py
        moduleB.py
tests/
    __init__.py
    test_module1.py
    test_module2.py
    subpackage1/
        __init__.py
        test_moduleA.py
        test_moduleB.py

This way, you can just run nosetests in the toplevel directory and all tests will be found. You need to make sure that src/ is on the PYTHONPATH, however, otherwise all the tests will fail due to missing imports.

score 8 · Accepted Answer · answered Dec 16 '08 at 23:28

8

If they all begin with test then just nosetest should work. Nose automatically searches for any files beginning with 'test'.

answered Dec 16 '08 at 23:28

Singletoned

5,089
3
30
32

3

I found the problem, nosetests skip they because are executables. I fixed resetting the executable permission bit and removing svn:executable property from subversion property. – Andrea Francia Dec 17 '08 at 19:34
3

The [--exe](http://packages.python.org/nose/usage.html#cmdoption--exe) option will prevent nosetests from skipping executable files. – Jesse Merriman Sep 28 '11 at 00:50
It is probably worth considering the move to `nose2` instead of `nosetest`. https://nose2.readthedocs.io/en/latest/getting_started.html – pds Jun 08 '17 at 08:45

score 6 · Answer 3 · answered Dec 15 '08 at 08:15

I don't know about nosetests, but you can achieve that with the standard unittest module. You just need to create a test_all.py file under your root directory, then import all your test modules. In your case:

import unittest
import test_module1
import test_module2
import subpackage1
if __name__ == "__main__":
    allsuites = unittest.TestSuite([test_module1.suite(), \
                                test_module2.suite(), \
                                subpackage1.test_moduleA.suite(), \
                                subpackage1.test_moduleB.suite()])

each module should provide the following function (example with a module with two unit tests: Class1 and Class2):

def suite():
    """ This defines all the tests of a module"""
    suite = unittest.TestSuite()
    suite.addTest(unittest.makeSuite(Class1))
    suite.addTest(unittest.makeSuite(Class2))
    return suite
if __name__ == '__main__':
   unittest.TextTestRunner(verbosity=2).run(suite())

This is what I was looking for, except that I think you should omit the `()` after each suite name (at least under Python 2.6.5). — Trevor Burnham, Jun 21 '10 at 19:55
Python 2.7 has a way to automatically find test modules/packages, which seems easier to maintain than remembering to add them explicitly to a test_all.py. See http://docs.python.org/library/unittest.html#test-discovery — Eric Smith, Oct 28 '11 at 15:54

score 2 · Answer 4 · answered Dec 14 '08 at 17:37

This is probably a hotly-contested topic, but I would suggest that you separate your tests out from your modules. Set up something like this...

Use setup.py to install these into the system path (or you may be able to modify environment variables to avoid the need for an "install" step).

foo/
    module1.py
    module2.py
    subpackage1/
        __init__.py
        moduleA.py
        moduleB.py

Now any python script anywhere can access those modules, instead of depending on finding them in the local directory. Put your tests all off to the side like this:

tests/
    test_module1.py
    test_module2.py
    test_subpackage1_moduleA,py
    test_subpackage2_moduleB.py

I'm not sure about your nosetests command, but now that your tests are all in the same directory, it becomes much easier to write a wrapper script that simply imports all of the other tests in the same directory. Or if that's not possible, you can at least get away with a simple bash loop that gets your test files one by one:

#!/bin/bash
cd tests/
for TEST_SCRIPT in test_*.py ; do
    nosetests -m $TEST_SCRIPT
done

I'd suggest to reflect the package structure of the program in the test directory. Otherwise the test directory will become a mess when the program grows. — deamon, Mar 17 '11 at 21:45

score 0 · Answer 5 · answered Dec 14 '08 at 18:32

I'll give a Testoob answer.

Running tests in a single file is like Nose:

testoob test_foo.py

To run tests in many files you can create suites with the Testoob collectors (in each subpackage)

# src/subpackage?/__init__.py
def suite():
  import testoob
  return testoob.collecting.collect_from_files("test_*.py")

and

# src/alltests.py
test_modules = [
    'subpackage1.suite',
    'subpackage2.suite',
]

def suite():
    import unittest
    return unittest.TestLoader().loadTestsFromNames(test_modules)

if __name__ == "__main__":
    import testoob
    testoob.main(defaultTest="suite")

I haven't tried your specific scenario.

How to organize python test in a way that I can run all tests in a single command?

5 Answers5

Linked