2

Questions about this issue have been asked many times but I'm afraid I still cannot fathom why a module which imports from a subdirectory, works when I call the module from its own directory, but when I import it from another directory I get ModuleNotFoundError: No module named _nested_package. Here is the exact situation:

tbrowne@nd2110:~/scratch$ tree
.
└── topmodule
    ├── __init__.py
    ├── main.py
    ├── nest
    │   ├── __init__.py
    │   └── nestmodule.py
    └── topmodule.py

now here is main.py:

from topmodule import test_add_two

if __name__ == "__main__":
    print(test_add_two(11, 12))

here is topmodule.py:

from nest.nestmodule import add_two

def test_add_two(x, y):
    return add_two(x, y)

if __name__ == "__main__": 
    print(test_add_two(1, 2))

And here is nestmodule.py:

def add_two(x, y):
    return x + y

Now if I run main.py, from its own directory, no problem:

tbrowne@nd2110:~/scratch/topmodule$ python3 main.py
23

however if I move to the directory above, say scratch, and create nesttest.py:

tbrowne@nd2110:~/scratch$ cat nesttest.py
from topmodule.topmodule import test_add_two

if __name__ == "__main__":
    print(test_add_two(3, 4))

Then I run it:

tbrowne@nd2110:~/scratch$ python3 nesttest.py 
Traceback (most recent call last):
  File "/home/tbrowne/scratch/nesttest.py", line 1, in <module>
    from topmodule.topmodule import test_add_two
  File "/home/tbrowne/scratch/topmodule/topmodule.py", line 1, in <module>
    from nest.nestmodule import add_two
ModuleNotFoundError: No module named 'nest'

Issue is, I'm going to want to package this with Poetry or some other package creator, and I want topmodule from wherever it is imported, even if it is packaged up, to be able to import the nest.nestmodule's functions.

I'm hoping that by being very explicit on the exact situation, someone can help me on how to do this. The wider context in the real project I'm working on (not this toy analogous example) is that I am using git submodules inside other git repos, and these will have to be packaged up along with the main repo.

Thomas Browne
  • 23,824
  • 32
  • 78
  • 121

2 Answers2

2

You have three separate but intersecting issues here:

  1. Python 3 does not allow implicit relative imports, so from nest.nestmodule import add_two is automatically wrong since nest is not a top-level package.
  2. Once you fix that, it becomes wrong to run main.py as a file, since doing so puts its directory in sys.path, which does make nest a top-level package that doesn’t work with from .nest…. Putting it in a package implies that it is a module, so it must be run with -m. (There are still pitfalls here: everything about __main__ is broken to some degree.)
  3. You’re using submodules (presumably for nest), which are (opinion!) a bad idea in general and (fact) incompatible with any code, in nest or otherwise, which believes it to be a top-level package (see #1). “Fixing” this for code that you can’t edit will involve installing it as a dependency, not a subpackage, which is what you should be doing anyway.
Davis Herring
  • 36,443
  • 4
  • 48
  • 76
  • so, goodbye any pythonic way to use git submodules in a module? Sounds overly harsh, or is there some middle ground which you would recommend? – Thomas Browne Jan 29 '22 at 19:02
  • @ThomasBrowne: While not actually recommending submodules for other reasons, you can certainly use them *outside* of any package—and then either set up `sys.path` to refer to potentially several directories **containing** top-level packages or “build” your code and the submodules into a single installation directory to so use. – Davis Herring Jan 29 '22 at 22:48
0

use this and print import paths

import sys
print(sys.path)

you will see that only your current directory level is added to the path.

to solve your issue, you can do one of these:

  • add your sub directories to this import path list
  • use current folder's name as main module name like this:
    • from topmodule.nest.nestmodule import add_two
  • use nesting/submodule level with .:
    • from .nest.nestmodule import add_two
Yılmaz Durmaz
  • 2,374
  • 12
  • 26
  • solution 1) is kinda clunky so no. Solutions 2 and 3 only work if you're OUTSIDE the topmodule directory, and in case of package installation, solution 3 is actually quite good. So that's the way I'll go. Still, I'm going to leave the question open a bit longer to see if there are any other opinions out there. – Thomas Browne Jan 30 '22 at 12:07