2

I defined a package that include a dynamically growing set of modules:

- mypackage
    - __init__.py
    - module1.py
    - module2.py
    - module3.py
    ... many more .py files will be added

I could expose every name in every module in __init__.py like this:

from module1 import *
from module2 import *
from module3 import *

Now when I import mypackage in client code, I get all the names defined in the sub-modules:

# suppose funcA is defined in module1, class B is defined in module2
import mypackage
mypackage.funcA() # call module1.funcA
b = mypackage.B() # module2.B

The problem is, I could define many new modules in mypackage, and I don't want to add an extra line from modulex import * to __init__.py, every time I add a new module to the package.

What is the best way to dynamically export names in all submodules?

NeoWang
  • 17,361
  • 24
  • 78
  • 126

3 Answers3

4

In Python 3 the solution is straightforward using importlib and pkgutil.

Placing the following code in __init__.py is the same typing from submodule import * for all submodules (even nested ones).

import importlib
import pkgutil

for mod_info in pkgutil.walk_packages(__path__, __name__ + '.'):
    mod = importlib.import_module(mod_info.name)

    # Emulate `from mod import *`
    try:
        names = mod.__dict__['__all__']
    except KeyError:
        names = [k for k in mod.__dict__ if not k.startswith('_')]

    globals().update({k: getattr(mod, k) for k in names})

If you only want to include immediate submodules (e.g. pkg.mod but not pkg.mod.submod), replace walk_packages with iter_modules.

I emulated from mod import * based on this answer: How to do from module import * using importlib?

axblount
  • 2,639
  • 23
  • 27
  • Great tip for updating `globals` automatically! In order to have the imports work as the OP requested, you also need to append to `__all__` like other answers have shown – Addison Klinke Apr 14 '21 at 17:37
3

I'm not sure if this is what you mean:

but if i've understood correctly - do this in your __init__.py file. 

import os

__all__ = []

for module in os.listdir(os.path.dirname(__file__)):
    if module != '__init__.py' and module[-3:] == '.py':
        __all__.append(module[:-3])

You're adding all files in the same package into the __all__

Rcynic
  • 392
  • 3
  • 10
  • But this doesn't import the submodules. Perhaps I didn't make myself clear, I have updated the question. – NeoWang Sep 06 '15 at 07:50
  • This solution will let you do this `from myPackage import module1` and then `module1.funcA()`. And you can add any number of modules into the package without changing `__init__.py` – Rcynic Sep 07 '15 at 03:38
2

Simply adding module names to __all__ will not always serve the purpose. I came across such an issue and also required them to be imported in addition to adding them to __all__. This is the code I came up with to make it work in my case. I didn't have any sub packages, so this code works only at the top level.

modules = glob.glob(os.path.dirname(__file__) + '/*.py')
__all__ = []
for mod in modules:
  if not str(mod).endswith('__init__.py'):
    package_prefix = __name__ + '.'
    module_name = str(mod)[str(mod).rfind('\\') + 1:-3]
    __all__.append(module_name)
    __import__(package_prefix + module_name, globals(), locals(), [''])
smonani
  • 123
  • 1
  • 5