How can I import python modules from outside the module's project structure without getting a ModuleNotFoundError?

Question

I created a python project, using Pycharm if it matters, that looks like this:

outer_dir/
    outside_main.py
    module_example/
        module_main.py
        tests/
            test_1.py
            …
        level_1/
            foo_1.py
            level_2/
                foo_2.py

main.py calls functions from foo_1.py foo_1.py calls functions from foo_2.py

The general use case I'm trying to emulate is a python "package" or repo called module_example that has been downloaded and used by outside_main.py at the same level as the top-level package directory.

Here's how module_main.py looks:

# module_main.py

from level_1.foo_1 import foo_function_1
from level_1.level_2.foo_2 import foo_function_2


def main():
    print(foo_1())
    print(foo_2())


if __name__ == "__main__":
    main()

And I can call this top level script from test_1.py just fine like this:

# test_1.py

from module_main import main

if __name__ == "__main__":
    main()

However, when I try to use main() from outside the module_example directory, with code like this:

# outside_main.py

from module_example.module_main import main

if __name__ == "__main__":
    main()

I get this error:

Traceback (most recent call last):
  File "/path/to/outside_main.py", line 1, in <module>
    from module_example.module_main import main
  File "/path/to/module_example/module_main.py", line 1, in <module>
    from level_1.foo_1 import foo_function_1
ModuleNotFoundError: No module named 'level_1'

What seems to be happening is that regardless of directory of the python script being imported, its own imports are being searched for relative to the directory from which the main script is executed.

I'm looking for a method of importing that works equally whether I'm calling outside_main.py or test_1.py, and regardless of what directory the terminal is currently pointing at.

I've read this post about how python imports work, and it basically suggests I'm out of luck, but that makes no sense to me, all I'm trying to do is emulate using someone's open source library from github, they usually have a tests folder within their repo, that works fine, but also I can just unzip the entire repo in my project and call import on their py files without issue. How do they do it?

Can you tell us why you *dont* want to make your code a package? This would make importing much easier for you. — H. Doebler, Aug 23 '21 at 11:43
@H.Doebler Your question is telling me there's something critical I don't understand about importing. I'm not against making anything a "package" as long as I can save everything in the `module_example` directory as a git repo, and move it into multiple projects without changing all the import statements each time. — CSStudent7782, Aug 23 '21 at 11:51
Maybe it is just a typo? You did `from module_example.modules_main import main` but your file is `module_main.py`. — H. Doebler, Aug 23 '21 at 12:02
in your `import` statement is an `s` that is not in your modules file name. — H. Doebler, Aug 23 '21 at 12:06
do I understand correctly, that `module_example` contains a bunch of code that you intend to use in multiple other project of yours? — H. Doebler, Aug 23 '21 at 12:13

Niel Godfrey Pablo Ponciano · Answer 1 · 2021-08-23T12:44:31.970

Solution 1:

Use explicit relative imports (relative to the files inside the directory module_example). Note that the usage of implicit relative imports has already been removed in Python3

Implicit relative imports should never be used and have been removed in Python 3.

outer_dir/module_example/module_main.py

# Note the "." in the beginning signifying it is located in the current folder (use ".." to go to parent folder)
from .level_1.foo_1 import foo_function_1
from .level_1.level_2.foo_2 import foo_function_2

# All the rest the same

Solution 2:

Include the path to module_example in your PYTHONPATH environment variable (or you can also choose to update sys.path directly)

export PYTHONPATH=$(pwd):$(pwd)/module_example

Output:

Both solutions above was successful for me.

Before the fix:

$ python3 outside_main.py 
Traceback (most recent call last):
  File "outside_main.py", line 1, in <module>
    from module_example.module_main import main
  File "/home/nponcian/Documents/PearlPay/Program/2021_07Jul_31_StackOverflow/1/module_example/module_main.py", line 1, in <module>
    from level_1.foo_1 import foo_function_1
ModuleNotFoundError: No module named 'level_1'

After the fix:

$ python3 outside_main.py 
This is foo_function_1
This is foo_function_2

Have u tested this for imports in `test_1.py` when it is called as a script (`python /path/to/outer_dir/module_example/tests/test_1.py`). — H. Doebler, Aug 23 '21 at 12:10
I would not recommend `sys.path` or `PYTHONPATH`-fiddling to anyone. Thats a real pain compared to `pip -e` in a venv. — H. Doebler, Aug 23 '21 at 12:20

H. Doebler · Accepted Answer · 2021-08-23T12:27:23.693

Before entering the hell of sys.path-patching, I would suggest to wrap your code that currently lives in module_example into a package_example directory, add a setup.py and throw a number of __init__.py in.

Make package_example a git repo.

In every other project where you want to use module_example.[...], you create a venv and pip -e /path/to/package_example.

outer_dir/
    outside_main.py
    package_example/
        .git/
            ...
        setup.py
        module_example/
            __init__.py
            module_main.py
            tests/
                ...

where __init__.py (one in each subdirectory that contains modules that are to be imported) can just be empty files and setup.py has to be filled according to the distutils documentation.

What are the file contents of `setup.py` and `__init__.py`? – CSStudent7782 Aug 23 '21 at 12:24 — CSStudent7782, Aug 23 '21 at 12:24

How can I import python modules from outside the module's project structure without getting a ModuleNotFoundError?

2 Answers2

Solution 1:

Solution 2:

Output: