1

I have the following directory structure:

.
├── main.py
└── subfolder1
    ├── file1.py # def hello(): print("hello")
    ├── __init__.py 
    └── subfolder2
        ├── file2.py # def world(): print("world")
        └── __init__.py

I'm trying to be to access functions from both subfolder1/file.1py and subfolder/subfolder2/file2.py.

I have an empty __init__.py in each nested folder as per the documentation I could find. I'm assuming that's necessary. (Update: I've just now removed the __init__.py's from both subfolders and it still works. I thought having this was necessary?)

Contents of main.py:

import subfolder1.file1
import subfolder1.subfolder2.file2

subfolder1.file1.hello()
subfolder1.subfolder2.file2.world()

When I execute main.py, both functions are called correctly. But the syntax is more cumbersome than I'd like, I don't want to have to repeatedly specify directory paths in subsequent imports.

I've tried doing something like

import subfolder1 as sub1
import sub1.subfolder2 as sub2

sub1.file1.hello()
sub2.file2.world()

But my IDE's Pylance alerts me that Import "sub1.subfolder2" could not be resolved and the terminal outputs on execution

Traceback (most recent call last):
  File "/home/me/project/main.py", line 2, in <module>
    import sub1.subfolder2 as sub2
ModuleNotFoundError: No module named 'sub1'

I've seen online the sys.append('/path/to/subfolder') method, but I don't like having to take up a line of code before every multi-nested import. It defeats the purpose, as my goal is to avoid repeatedly typing out paths, whether in the import statement or a sys.append() method.

Are there any ways to do multi-level modularization more elgantly/succintly in Python?

Michael Moreno
  • 947
  • 1
  • 7
  • 24

1 Answers1

1

Nice Minimal Reproducible Example !

About __init__.py files, see this question, but the gist is :

  • They were required in Python2 and Python3.3 to indicate that a directory is a python package (from which you can import), but it is now optional (Python >= 3.4).
  • It defines what ends up in your module subfolder1 when you do import subfolder1.

See the Python language reference about packages.

Given your initial architecture, you can see what does your sub1 module contains/defines using dir :

# file: main.py
import subfolder1 as sub1
print(dir(sub1))
# ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']

As all of these are dunder methods related to the Python import machinery, it should not come as a surprise that sub1.subfolder2 fails : there is actually no subfolder2 in sub1.

Contrast with this example :

import logging
print(dir(logging))
# logging stuff :
#   [..., 'CRITICAL', 'DEBUG', 'ERROR', 'FATAL', 'Logger', 'getLogger', ...
# regular module/package/import stuff :
#    '__all__', '__author__', '__builtins__', '__cached__', '__date__', '__doc__', '__file__', '__loader__', '__name__',
#    '__package__', '__path__', '__spec__', '__status__', '__version__', ...
# and names of other modules :
#    'os', 're', ..., 'sys', 'threading', 'time', ...]

# which means I can do :
print(logging.DEBUG)
# 10
print(logging.__author__)
# Vinay Sajip <vinay_sajip@red-dove.com>
print(logging.sys.version)
# 3.9.10 (main, Jan 15 2022, 18:17:56)
# [GCC 9.3.0]
import sys
print(sys.version)
# 3.9.10 (main, Jan 15 2022, 18:17:56)
# [GCC 9.3.0]
print(logging.sys is sys)
# True

If you take a look at the implementation for your (C)Python library sys, for example using your IDE's "go-to-definition" you will get an __init__.py file (mine is /usr/lib/python3.9/logging/__init__.py). And it defines exactly what dir listed to me.

(I chose logging because it is implemented in pure Python, while other libraries may not)

Your __init__.py file defines what will be in subfolder1 when you do just import subfolder1.

How to control what ends up in your library ?

import logging
from logging import *
print(DEBUG)  # 10
print(sys.version)  # NameError: name 'sys' is not defined

because

import logging
print("DEBUG" in logging.__all__)  # True
print("sys" in logging.__all__)  # False

Citing this answer about __all__ :

It's a list of public objects of that module, as interpreted by import *. It overrides the default of hiding everything that begins with an underscore.

Now, to answer your question "how to import neatly" :

# file: subfolder1/__init__.py
from . import file1

# file: subfolder1/subfolder2/__init__.py
from . import file2

# file: main.py
import subfolder1 as sub1
import subfolder1.subfolder2 as sub2
sub1.file1.hello()
sub2.file2.world()

You can't import sub1.subfolder2 because Python won't find a sub1 package (unless by messing with loaders).
But you can import subfolder1.subfolder2 because there exist a directory subfolder1/subfolder2.

What you could also do is :

# file: subfolder1/__init__.py
from . import subfolder2
from . import file1

# file: main.py
import subfolder1 as sub1
sub1.file1.hello()
sub1.subfolder2.file2.world()

or else :

# file: subfolder1/__init__.py
from . import subfolder2 as sub2
from . import file1

# file: main.py
import subfolder1 as sub1
sub1.file1.hello()
sub1.sub2.file2.world()
Lenormju
  • 4,078
  • 2
  • 8
  • 22