24

Here's the structure I'm working with:

directory/
          script.py
          subdir/
                 __init__.py
                 myclass01.py
                 myclass02.py

What I want to do is import in script.py the classes defined in myclass01.py and myclass02.py. If I do:

from subdir.myclass01 import *

It works fine for the class defined in myclass01.py. But with this solution if there are many classes defined in different files in subdir and I want to import all of them, I'd have to type one line for each file. There must be a shortcut for this. I tried:

from subdir.* import *

But it didn't work out.

EDIT: here are the contents of the files:

This is __init__.py (using __all__ as Apalala suggested):

__all__ = ['MyClass01','MyClass02']

This is myclass01.py:

class MyClass01:
    def printsomething():
        print 'hey'

This is myclass02.py:

class MyClass02:
    def printsomething():
        print 'sup'

This is script.py:

from subdir import *
MyClass01().printsomething()
MyClass02().printsomething()

This is the traceback that I get when I try to run script.py:

File "script.py", line 1, in <module>
    from subdir import *
AttributeError: 'module' object has no attribute 'MyClass01'
martineau
  • 119,623
  • 25
  • 170
  • 301
liewl
  • 4,021
  • 13
  • 46
  • 65
  • 1
    I don't know how this is done, but don't do this. Explicitly import each module. You will run into all sorts of problems later if you do this. It's not worth it, imo. – user225312 Feb 27 '11 at 18:09
  • What kind of problems could this bring? – liewl Feb 28 '11 at 13:41
  • I don't know what sort of problems @user225312 is cautioning you about, but I have used code similar to that my answer on a number of occasions without encountering any issues. In fact, I think it's a fairly good way to avoid hardcoding of things into your scripts as well as also being a decent way to implement a plug-in software architecture. – martineau Oct 02 '12 at 16:46
  • 1
    I guess this link does it the best. http://stackoverflow.com/questions/4142151/python-how-to-import-the-class-within-the-same-directory-or-sub-directory – Soham Bhattacharya Aug 17 '13 at 21:46

4 Answers4

12

Although the names used there are different from what's shown in your question's directory structure, you could use my answer to the question titled Namespacing and classes. The __init__.py shown there would have also allowed the usepackage.py script to have been written this way (package maps to subdir in your question, and Class1 to myclass01, etc):

from package import *

print Class1
print Class2
print Class3

Revision (updated):

Oops, sorry, the code in my other answer doesn't quite do what you want — it only automatically imports the names of any package submodules. To make it also import the named attributes from each submodule requires a few more lines of code. Here's a modified version of the package's __init__.py file (which also works in Python 3.4.1):

def _import_package_files():
    """ Dynamically import all the public attributes of the python modules in this
        file's directory (the package directory) and return a list of their names.
    """
    import os
    exports = []
    globals_, locals_ = globals(), locals()
    package_path = os.path.dirname(__file__)
    package_name = os.path.basename(package_path)

    for filename in os.listdir(package_path):
        modulename, ext = os.path.splitext(filename)
        if modulename[0] != '_' and ext in ('.py', '.pyw'):
            subpackage = '{}.{}'.format(package_name, modulename) # pkg relative
            module = __import__(subpackage, globals_, locals_, [modulename])
            modict = module.__dict__
            names = (modict['__all__'] if '__all__' in modict else
                     [name for name in modict if name[0] != '_'])  # all public
            exports.extend(names)
            globals_.update((name, modict[name]) for name in names)

    return exports

if __name__ != '__main__':
    __all__ = ['__all__'] + _import_package_files()  # '__all__' in __all__

Alternatively you can put the above into a separate .py module file of its own in the package directory—such as _import_package_files.py—and use it from the package's __init__.py like this:

if __name__ != '__main__':
    from ._import_package_files import *  # defines __all__
    __all__.remove('__all__')  # prevent export (optional)

Whatever you name the file, it should be something that starts with an _ underscore character so it doesn't try to import itself recursively.

martineau
  • 119,623
  • 25
  • 170
  • 301
  • 7
    Wow that's a rather complex solution for this problem. – liewl Feb 28 '11 at 14:21
  • `__init__.py` is pretty generic, which tends to complicate things (and is why many just hard-code things into their programs). Fortunately it only executes once and its compiled byte-code is saved. I tested it before posting my answer and again before this comment. What seems to be the problem? – martineau Feb 28 '11 at 20:14
  • When I try to call a method with `MyClass01().printsomething()` in script.py it says that name MyClass01 is not defined. I guess the importing process is failing somehow. – liewl Feb 28 '11 at 21:00
  • @David McDavidson: If no exceptions are occurring, then it sounds like it's not finding your `MyClass01.py` file. `__init__.py` and `MyClass01.py` need to be in the same subdirectory. You could try putting a `print subpackage` statement right before the `try:` in `__init_.py` to see what .py files it's finding and processing. Also a `print dir()` in your `script.py` right after the `from/import` statement would show what, if anything, was imported. – martineau Feb 28 '11 at 21:36
  • Ok, I added `print subpackage` in `__init__.py` and `print dir()` in `script.py`. Running `script.py` printed out `subdir.myclass01 subdir.myclass02`, `['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'myclass01', 'myclass02']`, and the 'MyClass01 is not defined' error. – liewl Feb 28 '11 at 22:03
  • It worked. Is there a way to not have to use `myclass01.` preceding it? – liewl Feb 28 '11 at 23:01
  • @David McDavidson: Yes, see my revised answer which will also import `myclass01.MyClass01` in the namespace (not just `myclass0`). – martineau Feb 28 '11 at 23:31
  • @David McDavidson: Glad to hear that. Please note that as currently written ignores any `__all__` attribute of modules and packages which normally determines what names are public and get imported when `*` is used. It's would be possible to do this, but doing so would likely complicate the code further. – martineau Mar 01 '11 at 02:32
  • @DavidMcDavidson: Ah, heck, making it also honor `__all__` isn't that much more complicated. – martineau Mar 18 '13 at 23:16
  • Sorry `martineau`, I copied your code exactly in the 2 `__init__.py` files that I have in the 2 subdirectories, but I am still having import errors: `Traceback (most recent call last): File "/Users/cell/Desktop/xo/main.py", line 4, in from logic.xologic import * File "/Users/cell/Desktop/xo/logic/__init__.py", line 32, in __all__ = ['__all__'] + _import_package_files() # '__all__' in __all__ File "/Users/cell/Desktop/xo/logic/__init__.py", line 22, in _import_package_files` – nbro Nov 30 '14 at 12:46
  • Continues here: `module = __import__(subpackage, globals_, locals_, [modulename]) File "/Users/cell/Desktop/xo/logic/bot.py", line 12, in from board import Board ImportError: No module named 'board'` – nbro Nov 30 '14 at 12:46
  • @nbro: My code seems to still work in Python 3.4.1, so I have no reason to think that's the problem. It's very difficult to determine what the issue is from a traceback you tried to stuff into some comments. Also, since generally a package only has one `__init__.py`, I don't quite understand what you meant by "in the 2 `__init__.py` files that I have". I suggest that you post a separate question describing your problem in detail and put the traceback in it. – martineau Nov 30 '14 at 23:50
  • Thanks for the answer, for of all. I had more than 1 folder, each of them having a `__init__.py` file. Now I solved fortunately the problem. I understood that `__init__.py` basically tells the `Python` interpreter that the folder where `__init__.py` is a namespace (right?). So I decided to change the imports and instead of saying `from board import Board`, where `board.py` is a script under the namespace (folder) `logic`, I import it in this way: `from logic.board import Board`, and now it seems to work, even with a blank `__init__.py` file – nbro Dec 01 '14 at 00:30
  • Unfortunately, I cannot post question anymore, because some of my previous questions where strongly downvoted (unfairly, in my opinion). Is there a possibility, if you know, to have the possibility to post again? (sorry the for the pun) – nbro Dec 01 '14 at 00:33
  • @nbro: Actually I did find something that had to be changed to get it work in Python 3.4.1. See the comments on my last revision to the answer. – martineau Dec 01 '14 at 12:06
6

Your best option, though probably not the best style, is to import everything into the package's namespace:

# this is subdir/__init__.py
from myclass01 import *
from myclass02 import *
from myclass03 import *

Then, in other modules, you can import what you want directly from the package:

from subdir import Class1
Apalala
  • 9,017
  • 3
  • 30
  • 48
  • @David McDavidson It could be good style. It's just that one should always ask oneself if there isn't a better solution to overpopulating a namespace. Note that from the original question it seems that the author is defining a single class per Python module, something which is not very pythonic. – Apalala Feb 28 '11 at 14:27
  • 1
    Originally I had all the classes in a single file. Since the classes were growing larger, I separated them in different files and put them in a directory. What would be the pythonic way to do this? – liewl Feb 28 '11 at 15:58
  • @David McDavidson Better control over source code is a valid requirement, but one should aim to maintain a logical module structure from the point of view of its users. If you want to use the approach I suggest, I would rename the internal modules to the likes of `__myclass01.py` so clients of the library are aware that they shouldn't use those modules because they are private. I would also use `__all__` in `__init__.py`. – Apalala Feb 28 '11 at 16:51
  • Ok, using the solution you provided on your answer was working. Then I commented out the `from ... import *` statements and added `__all__ = ['myclass01','myclass02']`. Calling `from subdir import *` in script.py should work, right? Its not working. – liewl Feb 28 '11 at 19:11
  • @David McDavidson Please provide a more specific example. Your original question doesn't mention the class names. At any rate, `__all__` should contain the class names, not the private module names. – Apalala Feb 28 '11 at 22:31
  • I updated the question with the class definitions and the resulting traceback. – liewl Feb 28 '11 at 22:55
  • Try `import subdir` and then `print dir(subdir)` with, and without the `__all__` definition. I just tried your recipe, and it worked on Python 2.6. The disadvantage of `__all__` is that it uses strings, so typos may go unchecked. – Apalala Mar 03 '11 at 05:09
  • From the error message, it seems you omitted adding `from myclass01 import *` to `subdir/__init__.py` – Apalala Mar 03 '11 at 05:20
6

I know it's been a couple months since this question was answered, but I was looking for the same thing and ran across this page. I wasn't very satisfied with the chosen answer, so I ended up writing my own solution and thought I'd share it. Here's what I came up with:

# NOTE: The function name starts with an underscore so it doesn't get deleted by iself
def _load_modules(attr_filter=None):
    import os

    curdir = os.path.dirname(__file__)
    imports = [os.path.splitext(fname)[0] for fname in os.listdir(curdir) if fname.endswith(".py")]

    pubattrs = {}
    for mod_name in imports:
        mod = __import__(mod_name, globals(), locals(), ['*'], -1)

        for attr in mod.__dict__:
            if not attr.startswith('_') and (not attr_filter or attr_filter(mod_name, attr)):
                pubattrs[attr] = getattr(mod, attr)

    # Restore the global namespace to it's initial state
    for var in globals().copy():
        if not var.startswith('_'):
            del globals()[var]

    # Update the global namespace with the specific items we want
    globals().update(pubattrs)

# EXAMPLE: Only load classes that end with "Resource"
_load_modules(attr_filter=lambda mod, attr: True if attr.endswith("Resource") else False)
del _load_modules # Keep the namespace clean

This simply imports * from all .py files in the package directory and then only pulls the public ones into the global namespace. Additionally, it allows a filter if only certain public attributes are desired.

Justin Warkentin
  • 9,856
  • 4
  • 35
  • 35
0

I use this simple way:

  1. add the directory to the system path, then
  2. import module or from module import function1, class1 in that directory.

notice that the module is nothing but the name of your *.py file, without the extension part.

Here is a general example:

import sys
sys.path.append("/path/to/folder/")
import module # in that folder

In your case it could be something like this:

import sys
sys.path.append("subdir/")
import myclass01
# or
from myclass01 import func1, class1, class2 # .. etc
Aziz Alto
  • 19,057
  • 5
  • 77
  • 60