How do I import all the submodules of a Python namespace package?

Question

A Python namespace package can be spread over many directories, and zip files or custom importers. What's the correct way to iterate over all the importable submodules of a namespace package?

Answer 1

Here is a way that works well for me. Create a new submodule all.py, say, in one of the packages in the namespace.

If you write

import mynamespace.all

you are given the object for the mynamespace module. This object contains all of the loaded modules in the namespace, irrespective of where they were loaded, since there is only one instance of mynamespace around.

So, just load all the packages in the namespace in all.py!

# all.py

from pkgutil import iter_modules

# import this module's namespace (= parent) package
pkg = __import__(__package__)

# iterate all modules in pkg's paths,
# prefixing the returned module names with namespace-dot,
# and import the modules by name
for m in iter_modules(pkg.__path__, __package__ + '.'):
    __import__(m.name)

Or in a one-liner that keeps the all module empty, if you care about that sort of thing:

# all.py

(lambda: [__import__(_.name) for _ in __import__('pkgutil').iter_modules(__import__(__package__).__path__, __package__ + '.')])()  # noqa

After importing the all module from your namespace, you then actually receive a fully populated namespace module:

import mynamespace.all

mynamespace.mymodule1  # works
mynamespace.mymodule2  # works
...

Of course, you can use the same mechanism to enumerate or otherwise process the modules in the namespace, if you do not want to import them immediately.

Answer 2

Please read import confusion.

It very clearly distinguishes all the different ways you can import packages and its sub modules and in the process answers your question. When you need a certain submodule from a package, it’s often much more convenient to write from io.drivers import zip than import io.drivers.zip, since the former lets you refer to the module simply as zip instead of its full name.

from modname import *, this provides an easy way to import all the items from a module into the current namespace; however, this statement should be used sparingly.