6

I want to create a Python module that works like NumPy. The methods are not only sub-modules in the leaves of the tree from the module source. There is a root module containing many methods that I can call directly, and there are also sub-modules. The problem is the root methods must be defined somewhere. I was thinking to have a directory structure:

module/
  __init__.py
  core.py
  stuff1.py
  submodule/
    __init__.py
    stuff2.py
    stuff3.py

Now that I want is for everything inside "core" to be imported into the "module" namespace, as if it were a module.py file, and the contents of core.py were inside this module.py. The problem is that module is a directory instead of a file, so how do I define these methods that should sit in the root of the module?

I tried putting "from core import *" inside init.py, but that didn't work. (EDIT: Actually it does.)

Should I have the core methods inside a "module.py" file, and also a module directory? I don't know if that works, but it looks pretty awkward.

dividebyzero
  • 2,190
  • 1
  • 21
  • 33
  • 2
    What do you mean by "I tried putting "from core import *" inside init.py, but that didn't work."? What exactly did you try, and what went wrong? You ought to be able to put something like `from module.core import Clazz print Clazz()` into `module/__init__.py` and see the output (something like ``) when you import the module. Also remember that `from module import *` does not import any names that start with an underscore. – Henry Keiter Mar 11 '13 at 20:19
  • Henry, that works, but after the import you will get "NameError: name 'Clazz' is not defined". I want the contents of core.py to be loaded inside "module" after I do "import module". Importing core inside __init__ (e.g. import module.core as module) appears to work for whatever else is inside the init script, but not in the shell where I imported "module". – dividebyzero Mar 11 '13 at 22:29
  • I added an answer to clarify. – Henry Keiter Mar 12 '13 at 00:00

2 Answers2

5

What I think you want is to be able to do this:

# some_other_script.py

import module
# Do things using routines defined in module.core

What happens when you ask Python to import module is (in a very basic sense), module/__init__.py is run, and a module object is created and imported into your namespace. This object (again, very basically) encompasses the things that happened when __init__.py was run: name definitions and so on. These can be accessed through module.something.

Now, if your setup looks like this:

# module/__init__.py

from module.core import Clazz
c = Clazz()
print c # Note: demo only! Module-level side-effects are usually a bad idea!

When you import module, you'll see a print statement like this:

<module.core.Clazz object at 0x00BBAA90>

Great. But if you then try to access c, you'll get a NameError:

# some_other_script.py
import module # prints "<module.core.Clazz object at 0x00BBAA90>"
print c # NameError (c is not defined)

This is because you haven't imported c; you've imported module. If instead your entry-point script looks like this:

# some_other_script.py
import module # prints "<module.core.Clazz object at 0x00BBAA90>"
print module.c  # Access c *within module*

Everything will run fine. This will also work fine with from core import * and/or from module import *, but I (and PEP8) advise against that just because it's not very clear what's going on in the script when you start mucking around with wild imports. For clarity:

# module/core.py

def core_func():
    return 1


# module/__init__.py

from core import *
def mod_func():
    return 2

The above is really pretty much fine, although you might as well make core "private" (rename to _core) to indicate that there's no reason to touch it from outside the package anymore.

# some_other_script.py

from module import *

print core_func() # Prints 1
print mod_func() # Prints 2
Henry Keiter
  • 16,863
  • 7
  • 51
  • 80
3

Check out information about the __all__ list. It allows you to define what names are exported.

Tag it as such and you can setup a function to determine what to pull in from your submodules:

@property
all(self):
    #Whatever introspective code you may want for your modules
    __all__ += submodule.__all__

If you just want the whole damn shabang in module space, here's a way:

$ ipython
In [1]: from foomod import *

In [2]: printbar()
Out[2]: 'Imported from a foreign land'

In [3]: ^D
Do you really want to exit ([y]/n)?
$ ls foomod/
__init__.py __init__.pyc    core.py         core.pyc        submodule
$ grep . foomod/*.py 
foomod/__init__.py:from foomod.core import *
foomod/core.py:def printbar():
foomod/core.py:     return "Imported from a foreign land"

... and if we make __init__.py empty:

$ echo > foomod/__init__.py
$ ipython

In [1]: from foomod import *

In [2]: printbar()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-ba5b6693441e> in <module>()
----> 1 printbar()

NameError: name 'printbar' is not defined
Community
  • 1
  • 1
Jeff Ferland
  • 17,832
  • 7
  • 46
  • 76
  • Hi, Jeff. I'm not sure what you mean, what else do I have to do for that code to work? Is this all that is supposed to go inside module/__init__.py? The documentation says specifically that __all__ relates to "from module import *"... It actually lists all that should be automatically imported, then? It's not just when you use "from xxx *"? That is a pretty bad way of explaining things in the documentation then. Anyway, I want the contents of core to be part or the module namespace, not just import module.core automatically when module is imported. – dividebyzero Mar 11 '13 at 22:37