how to make main a package -OR- script and a package submodule at the same time

Question

I have this structure:

app.py
sub/
    __init__.py
    lib.py
    utils.py

In app.py, I want to be able to do sth like this:

from sub.lib import some_func

In lib.py, I want to be able to import utils.py.

And I also want that it's possible to execute lib.py as a script directly. (I.e. cd sub; python3 lib.py.)

The problem is, if I start lib.py as a script, it will not be a package, and thus, it cannot use relative imports, so sth like from .utils import some_util_func would not work. However, if I import lib.py as part of the sub package, only the relative import will work.

How do I solve this? (In an easy way. E.g. without creating another small wrapper script just to call lib.py.)

E.g. if there is a way to mark the __main__ module as a package, it would solve that. But how? Is this possible? E.g. by defining __path__ or so?

I would argue that running a module as a script is the wrong thing to do most of the time. If you something that can run as a script, create a separate script and import `lib`. — chepner, Feb 07 '19 at 15:45
@chepner: I think I have a valid case (it's just a simple demo, which runs `some_func` directly). Maybe just a matter of taste anyway. But I don't really understand why it's wrong to do that (except that I run into the problem described here in the question). Or asked differently: What's the problem with it (and with the current solution I wrote in my own answer)? — Albert, Feb 07 '19 at 16:38
The problem is that modules and scripts have two very different use cases, and the `if __name == "__main__"` hack to support both with the same file only works in simple cases. — chepner, Feb 07 '19 at 16:48
@chepner In what case does it not work? I don't really see what problem would be with this. (Except maybe that this is an uncommon thing to do, at least for now.) — Albert, Feb 07 '19 at 16:57
Well, for starters, relative imports break without explicit `PYTHONPATH` hacking... — chepner, Feb 07 '19 at 17:01

Albert · Accepted Answer · 2022-09-01T11:51:37.327

Ok, actually, my last comment really worked. If I add this in the beginning of lib.py, it seems to work:

import os

if __name__ == '__main__':
    __path__ = [os.path.dirname(os.path.abspath(__file__))]

(About __path__, see e.g. here.)

Edit With newer Python versions (3.9), this does not seem to be enough anymore. See below for an extended version.

Also, in PEP 366 it is mentioned:

When the main module is specified by its filename, then the __package__ attribute will be set to None. To allow relative imports when the module is executed directly, boilerplate similar to the following would be needed before the first relative import statement:

if __name__ == "__main__" and __package__ is None:

    __package__ = "expected.package.name"

I didn't tried this yet.

For Python 3.9, I needed to extend my solution:

if __name__ == '__main__':
    # Make relative imports work here. https://stackoverflow.com/questions/54576879/
    __path__ = [os.path.dirname(os.path.abspath(__file__))]
    __package__ = "YOUR_CUSTOM_PACKAGE_NAME"
    pkg_mod = types.ModuleType(__package__)
    sys.modules[__package__] = pkg_mod
    pkg_mod.__package__ = __package__
    pkg_mod.__file__ = os.path.abspath("__init__.py")
    pkg_mod.__path__ = __path__

score 0 · Answer 2 · answered Feb 07 '19 at 15:51

0

You can set PYTHONPATH environment variable when starting your program.

Just type from lib directory:

PYTHNONPATH="path/to/root/directory" python lib.py

answered Feb 07 '19 at 15:51

butusk0

1

No, that will not work here, because still `__main__` is not a package, so relative imports will not work. And also, I want that I can run the script as-is, without setting some env-var in the right way beforehand. – Albert Feb 07 '19 at 16:34

how to make __main__ a package -OR- script and a package submodule at the same time

2 Answers2

how to make main a package -OR- script and a package submodule at the same time