Importing a lot of modules from custom package

Question

Say i have this this structure.

MyApp
├── main.py
└── package
    ├── __init__.py
    ├── a.py
    ├── b.py
    ├── c.py
    ├── d.py
    ├── e.py
    ├── f.py
    ├── g.py
    ├── h.py
    ├── ...
    └── z.py

And in main.py I need to use all modules, from a.py to z.py

I'd like to know how I can import all those modules with one import statement.

So instead of doing

from package import a
from package import b
...
from package import z

I could just import the package and have all the modules ready.

Things I've tried

import package
a = package.a.A()
# AttributeError: 'module' object has no attribute 'a'

Now I know I could put a code in __init__.py to add all the modules to __all__, but from what I've read, we should avoid 'from package import *'

The reason for this is that the package might have an increasing number of modules and I would like to adding an import statement to the main code each time a module is created. Ideally I'd like to be able to just drop the module in the package and have it ready for use.

You can also look at the `__all__` variable [here](http://stackoverflow.com/questions/44834/can-someone-explain-all-in-python), maybe you can use it in your `__init__.py` — Amr, Jun 14 '12 at 19:21
@Amr this allow you to `from package import *` to get all the packages. — corn3lius, Jun 14 '12 at 19:50

score 4 · Accepted Answer · answered Jun 14 '12 at 19:05

4

In __init__.py, you can:

import a, b, c, d...

and then the modules will be placed in the package namespace after you do import package.

You you really want to names a, b, etc. in main.py's namespace, and have this happen with no effort, you can't really avoid from package import *; any other method of importing them all implicitly is going to be just as bad, since it involves polluting the namespace with names you don't explicitly import.

answered Jun 14 '12 at 19:05

Wooble

87,717
12
108
131

But i still wouldn't be able to just drop a module in the package directory and have it just work. Doing this way is a little different than doing 'from package import a' in main.py. Unless of course, i automate it, which is what im going to attempt right now. thanks for this. – sqram Jun 14 '12 at 19:10
Do you have a use case for having names injected into the main script's namespace that you don't even know about when writing the script? – Wooble Jun 14 '12 at 19:14
2

@lyrae the main difference is that you'd have to add the import once (in your package), instead of having to do all those imports in all applications using that package. – KurzedMetal Jun 14 '12 at 19:15

score 2 · Answer 2 · edited May 23 '17 at 10:34

2

What you propose is very bad design practice since you import all but not what is required. In general IT SLOWS DOWN program loading - never do it if not really required. Never initialize not used variables in modules also since it waste of time.

Two solution which not follow the good design practice if not used correctly.

Check this answer Can someone explain __all__ in Python?.

You could also use __import__ to load modules and os.path.dirname(__file__) to list all files names in directory and load as modules.

BTW this pattern is lead to serious security holes since you allow load anything - it need only creation permission to break security.

edited May 23 '17 at 10:34

Community

1
1

answered Jun 14 '12 at 19:18

Chameleon

9,722
16
65
127

Thanks for feedback. The thing is, all modules will always be used by the main program. – sqram Jun 14 '12 at 20:05
If you program always will use 100% of modules and has single run it is good pattern. Avoid it if you have even 100% cover since load on require is faster and eco than load all at once. – Chameleon Jun 15 '12 at 06:09

score 2 · Answer 3 · answered Jun 14 '12 at 19:18

2

I would recommend not doing this. If you must, this is the method I've used in the past:

# __init__.py

import os
import re

PACKAGE = 'MyApp.package'
MODULE_RE = r"^.*.py$"

for filename in os.listdir(os.path.dirname(__file__)):
  if not re.match(MODULE_RE, filename) or filename == "__init__.py":
    continue

  imported_module = __import__('%s.%s' % (PACKAGE, filename[:-3]),
                               {}, {},
                               filename[:-3])

answered Jun 14 '12 at 19:18

jterrace

64,866
22
157
202

1

I'd use the [`glob` module](http://docs.python.org/library/glob.html) for matching module filenames here instead of a regular expression. Even an `.endswith('.py')` would be better.. – Martijn Pieters Jun 14 '12 at 20:28

score 1 · Answer 4 · answered Jun 14 '12 at 19:11

1

This code is not very beautiful, but I think it is a nice workaround

import os

for i in os.listdir('package'):
    if i.endswith('.py') and not i.startswith('__'):
        exec('from package import ' + i[:-3])

answered Jun 14 '12 at 19:11

BrtH

2,610
16
27

2

Better is to use __import__ as explained in another answer and little fix filter on names. – Chameleon Jun 14 '12 at 19:25
1

And @Chameleon meant `__import__` of course, the `__` double underscore got interpreted as bold by SO there. – Martijn Pieters Jun 14 '12 at 20:23
@MartijnPieters yeah, I understood that. This answer was just a simple thought of me, and at that time there were no other answers. But you're right, `__import__` is probably better (but this is simpler :D) – BrtH Jun 14 '12 at 20:25

Importing a lot of modules from custom package

4 Answers4