3

Run python main.py where main.py has relative imports will fail.

<root>
└── src                            
   ├── main.py    # from main.py, want to use the stuff in other.py
   └── other.py

$ python main.py
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    from . other import use_me
ImportError: attempted relative import with no known parent package
  • main.py
# 'import ...' is for absolute import only
# For relative import, must be 'from ... import ...'
# See https://www.python.org/dev/peps/pep-0328/#guido-s-decision
from . other import use_me
if __name__ == "__main__":
    use_me()
  • other.py
def use_me():
    print("thanks")

PEP 338 -- Executing modules as scripts is clear that explicit relative imports don't work from a main module.

The release of 2.5b1 showed a surprising (although obvious in retrospect) interaction between this PEP and PEP 328 - explicit relative imports don't work from a main module. This is due to the fact that relative imports rely on __name__ to determine the current module's position in the package hierarchy. In a main module, the value of __name__ is always '__main__', so explicit relative imports will always fail (as they only work for a module inside a package).

Cause

The reason is explained as below and the workaround is to update __path__.

The issue isn't actually unique to the -m switch. The problem is that relative imports are based on __name__, and in the main module, __name__ always has the value __main__. Hence, relative imports currently can't work properly from the main module of an application, because the main module doesn't know where it really fits in the Python module namespace (this is at least fixable in theory for the main modules executed through the -m switch, but directly executed files and the interactive interpreter are completely out of luck).

You should be able get something similar to the old implicit relative import behaviour by sticking the following at the top of your module (before doing any relative imports): This makes the relative import machinery treat your main module as a package. The problem with this workaround is that, just like the old situation with implicit relative imports from the main module, you may end up with two different copies of the sibling modules in sys.modules. One copy will be __main__.foo' while the other will be 'package.foo' (with implicit relative imports, the first copy would have been a top level module called 'foo').

if __name__ = '__main__':
   from os.path import dirname
   __path__ = [dirname(__file__)]
   del dirname

Relative Imports and __name__

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

PEP 366 seems to have enabled the workaround to execute the main.py as a module.

The main.py can be executed via: python -m <parent_directory_name>.main.

This PEP proposes a backwards compatible mechanism that permits the use of explicit relative imports from executable modules within packages. Such imports currently fail due to an awkward interaction between PEP 328 and PEP 338.

By adding a new module level attribute, this PEP allows relative imports to work automatically if the module is executed using the -m switch. A small amount of boilerplate in the module itself will allow the relative imports to work when the file is executed by name.

The major proposed change is the introduction of a new module level attribute, package. When it is present, relative imports will be based on this attribute rather than the module name attribute.

As with the current name attribute, setting package will be the responsibility of the PEP 302 loader used to import a module. Loaders which use imp.new_module() to create the module object will have the new attribute set automatically to None. When the import system encounters an explicit relative import in a module without package set (or with it set to None), it will calculate and store the correct value (name.rpartition('.')[0] for normal modules and name for package initialisation modules). If package has already been set then the import system will use it in preference to recalculating the package name from the name and path attributes.

Question

Having read 1510172 and PEP 366, still could not understand the mechanism exactly how the error happens.

I suppose there is a module loader which uses __name__ which looks representing the module hierarchy. It looks it starts with <root> in the folder hierarchy below and ends with the module name itself, but end with __main__ for a main module run by python <script>.py.

<root>
└── src                            
   ├── main.py    # from main.py, want to use the stuff in other.py
   └── other.py

How these __name__, __path__,__package__, module loader are related, and what exactly is happening under the hood and how it causes the issue?

mon
  • 18,789
  • 22
  • 112
  • 205

1 Answers1

0

Found the answer in Relative imports in Python 3.

Note: The patch from the issue 18018 has added another if block, which will be executed before the code above:

if (PyUnicode_CompareWithASCIIString(package, "") == 0) {
    PyErr_SetString(PyExc_ImportError,
            "attempted relative import with no known parent package");
    goto error;
}

Which is now at cpython/blob/master/Python/import.c.

    last_dot = PyUnicode_GET_LENGTH(package);
    if (last_dot == 0) {
        goto no_parent_error;
    }

So what I understood is that if __name__ is set to '__main__' with no dot, it just goes to "attempted relative import with no known parent package".

The answer also explains:

we can run the module with the -m command line option that will "search sys.path for the named module and execute its contents as the __main__ module*"

... -m does all the importing stuff for you and automatically sets __package__, but you can do that yourself

mon
  • 18,789
  • 22
  • 112
  • 205