Running Unit Tests in Test Bundle in Python

Question

I am trying to write a parser in Python for a special text file format. To get an idea how to structure the code I looked into the source code of the JSON parser which is part of the Python standard library (Python/Lib/json).

In this json directory there is a tests directory which holds a number of unit tests. I replaced the json tests with my tests but now I do not know how to call them.

Looking into the directory there is a __init__.py file making it a module and inside of this file there is the following code snippet for running the tests:

here = os.path.dirname(__file__)

def test_suite():
    suite = additional_tests()
    loader = unittest.TestLoader()
    for fn in os.listdir(here):
        if fn.startswith("test") and fn.endswith(".py"):
            modname = "json.tests." + fn[:-3]
            __import__(modname)
            module = sys.modules[modname]
            suite.addTests(loader.loadTestsFromModule(module))
    return suite

def additional_tests():
    suite = unittest.TestSuite()
    for mod in (json, json.encoder, json.decoder):
        suite.addTest(doctest.DocTestSuite(mod))
    suite.addTest(TestPyTest('test_pyjson'))
    suite.addTest(TestCTest('test_cjson'))
    return suite

def main():
    suite = test_suite()
    runner = unittest.TextTestRunner()
    runner.run(suite)

if __name__ == '__main__':
    sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
    main()

My problem is now how are these unit tests executed? I am confused because the if __name__ == '__main__': if clause validates to true if this file is called directly and not being imported as a module. However as it is in the __init__.py file of the module it should be executed right after import.

Should an import tests in the python console start all the unit tests?

Answer 1

First: the __init__.py file in a directory marks the directory as a package and all other files with the .py ending therein are modules.

Second: To validate the __name__ == '__main__' conditional you have to execute the file. So a simple import won't do.

For further question on package and module structure i would suggest ro read the Official Python Documentation on Packages.

The unittests are structured into testsuites, which can contain one ore more TestCases which consists of one or more tests.

The tests are usually methods of a TestCase derived class. You can either run the tests by defining a runTest() method or defining multiple test_* methods, which will be executed autmatically. To execute a Unittest you can either use the convenience function unittest.main() which basicaly tries to construct a testsuite, testresult and testrunner object by using default-rules.

The execution of your unittest itself is done by the testrunner object. The standard testrunner-class is TextTestRunner, which uses a TextTestResult-class to store the testresults and prints them to stdout. See Official unittest Documentation for unittest.main() for the simplest variant.

Summary

1) A unittest is basically a TestSuite containing one ore more TestCases:

TestSuite      <---executed by--- TestRunner
   + TestCaseA                        |
       +test_a()                      |
       +test_b()             stores test-results 
           ...                      into
       +test_z()                      |
    + TestCaseB                       V
    + TestCaseC                  TestResult

2) The TestCases are subclasses of unittest.TestCase. You can create a TestSuite, by using a loader (e.g.: unittest.defaultTestLoader) which is basically a factory for a testsuite, or you can add the TestCases manually (suite.addTest(test) / suite.addTests(tests) - tests may be TestCases or even other TestSuites) or by combining those two methods.

3) To execute a testsuite you use a unittest.TestRunner-object which saves the results in a unittest.TestResult object.

Normaly you would be using unittest.TextTestRunner-object to get a simple output of the testresults to stdout.

That's excatly what happens also in your main-routine:

def main():
    suite = test_suite()                #1 create a TestSuite object
    runner = unittest.TextTestRunner()  #2 create a TextTestRunner object
    runner.run(suite)                   #3 executes the TestSuite with TestSuite
                                        #  build by the function test_suite()

To execute your testsuite you would have then to execute python __init__.py .