Using module's own objects in __main__.py

Question

I’m trying to access a module’s data from inside its __main__.py.

The structure is as follows:

mymod/
    __init__.py
    __main__.py

Now, if I expose a variable in __init__.py like this:

__all__ = ['foo']
foo = {'bar': 'baz'}

How can I access foo from __main__.py?

Answer 1

You need to either have the package already in sys.path, add the directory containing mymod to sys.path in __main__.py, or use the -m switch.

To add mymod to the path would look something like this (in __main__.py):

import sys
import os
path = os.path.dirname(sys.modules[__name__].__file__)
path = os.path.join(path, '..')
sys.path.insert(0, path)
from myprog import function_you_referenced_from_init_file

Using the -m switch would like:

python -m mymod

See this answer for more discussion.

Answer 2

The issue I run into the most with this type of thing is that I often want to run the __init__.py file as a script to test features, but these should not be run when loading the package. There is a useful workaround for the different execution paths between python <package>/__init__.py and python -m <package>.

• $ python -m <module> executes <package>/__main__.py. __init__.py is not loaded.
• $ python <package>/__init__.py simply executes the script __init__.py like a normal script.

The problem

When we want __init__.py to have an if __name__ == '__main__': ... clause that uses stuff from __main__.py. We can’t import __main__.py because it will always import __main__.pyc from the interpreter’s path. (Unless…we resort to absolute path import hacks, which can cause a lot of other mess).

The solution A solution :)

Use two script files for the module’s __main__:

<package>/
         __init__.py
         __main__.py
         main.py

---

# __init__.py

# ...
# some code, including module methods and __all__ definitions

__all__ = ['foo', 'bar']
bar = {'key': 'value'}
def foo():
    return bar
# ...
if __name__ == '__main__':
    from main import main
    main.main()

---

# __main__.py

# some code...such as:
import sys
if (len(sys.argv) > 1 and sys.argv[1].lower() == 'option1'):
    from main import main()
    main('option1')
elif (len(sys.argv) > 1 and sys.argv[1].lower() == 'option2'):
    from main import main()
    main('option2')
else:
    # do something else?
    print 'invalid option. please use "python -m <package> option1|option2"'

---

# main.py

def main(opt = None):
    if opt == 'option1':
        from __init__ import foo
        print foo()
    elif opt == 'option2':
        from __init__ import bar
        print bar.keys()
    elif opt is None:
        print 'called from __init__'

---

The imports in main.py are probably not ideal in the case we are running from __init__.py, as we are reloading them into the local scope of another module, despite having loading them in __init__.py already, but the explicit loading should avoid circular loading. If you do load the entire __init__ module again in your main.py, it will not be loaded as __main__, so should be safe as far as circular loading is concerned.

Answer 3

The __init__ module of a package acts like members of the package itself, so the objects are imported directly from mymod:

from mymod import foo

Or

from . import foo

if you like to be terse, then read about relative imports. You need to make sure, as always, that you do not invoke the module as mymod/__main__.py, for example, as that will prevent Python from detecting mymod as a package. You may wish to look into distutils.

Answer 4

If you run the module with python -m mymod then code in __main__.py will be able to import from the rest of the module without having to add the module to sys.path.

Answer 5

I found the first answer to be useful (i.e., hacking sys.path), but with the addition of pathlib in Python 3.4, I found the following code to be much more simple and Pythonic:

import sys
from pathlib import Path

# You don't need to .insert(), just append
sys.path.append(str(Path(__file__).parent.parent))

Answer 6

Module directory structure is as follows:

py/
   __init__.py
   __main__.py

__init__.py

#!/usr/bin/python3
#
# __init__.py
#

__all__ = ['foo']
foo = {'bar': 'baz'}
info = { "package": __package__,
         "name": __name__,
         "locals": [x for x in locals().copy()] }
print(info)

__main__.py

#!/usr/bin/python3
#
# __main__.py
#

info = { "package": __package__,
         "name": __name__,
         "locals": [x for x in locals().copy()] }
print(info)
from . import info as pyinfo
print({"pyinfo: ": pyinfo})

Execute the module as a script using the -m flag

$ python -m py

# the printout from the 'print(info)' command in __init__.py
{'name': 'py', 'locals': ['__all__', '__builtins__', '__file__', '__package__', '__path__', '__name__', 'foo', '__doc__'], 'package': None}
# the printout from the 'print(info)' command in __main__.py
{'name': '__main__', 'locals': ['__builtins__', '__name__', '__file__', '__loader__', '__doc__', '__package__'], 'package': 'py'}
# the printout from the 'print(pyinfo)' command in __main__.py
{'pyinfo: ': {'name': 'py', 'locals': ['__all__', '__builtins__', '__file__', '__package__', '__path__', '__name__', 'foo', '__doc__'], 'package': None}}