Python imports, packages, and runnable scripts

Question

I've read the following post:

It explains the difference between running a file as a top-level script and importing it as a module in remarkable clarity and detail.

I can't figure out, however, how to write the imports of a runnable script (if __name__ == '__main__' et cetera) in a way which I'd also be able to import it from my tests, all the while keeping my runnable scripts stand-alone (i.e. not requiring package installation).

Say I have the following project hierarchy:

/reporoot
    /mypkg
        /mysubpkg
            __init__.py
            subpkgmodule.py
        __init__.py
        main1.py
        main2.py
    /tests
        test_main1.py
        test_main2.py

test_main1 needs to import main1 in some manner, and main1 needs to import subpkgmodule in some manner. If, for instance, main1 imports subpkgmodule like so:

import mysubpkg.subpkgmodule

This will work fine when running the script as the top-level script because the top level script has its directory added to sys.path. This import will break, however, when the module is imported rather than run (because its directory will not be added to sys.path).

If main1 were to import subpkgmodule like so:

import mypkg.mysubpkg.subpkgmodule

This will only work when the package is an installed package (including python setup.py develop) and not running as a standalone script (i.e. chuck it on some filesystem and run python main1.py).

I saw that the standard library's http.server module has an import http.client and therefore cannot be run as a standalone script as I mentioned.

What would a clean solution for the imports of my runnable scripts be?

score 1 · Answer 1 · answered Dec 21 '15 at 21:50

Typical way would be to extend sys.path in runnable script with location of your package.

A list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default.

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.

A program is free to modify this list for its own purposes.

Sample test_main1.py:

import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), os.pardir))

import mypkg.mysubpkg.subpkgmodule   # should import fine

if __name__ == "__main__":
    pass  # runnable code here

I had thought about doing this via some `testenv` module that all the tests would import first, but it seemed inelegant — DoomMuffins, Dec 21 '15 at 21:55

score 0 · Answer 2 · answered Dec 21 '15 at 21:51

Libraries such as the built-in unittest have built-in test discovery. By default, it expects tests to be in files named with the pattern test*.py, where the asterisk (*) may be whatever you'd like. You'd then use discover and run the package using:

python -m unittest discover

This assumes that

you are using unittest,
your tests subclass unittest.TestCase, and
you use the default class unittest.TestLoader.

Python imports, packages, and runnable scripts

2 Answers2