Relative import problems in Python 3

Question

Python imports drive me crazy (my experience with python imports sometime doesn't correspond at all to idiom 'Explicit is better than implicit' :( ):

[app]
    start.py
        from package1 import module1
    [package1]
        __init__.py
            print('Init package1')
        module1.py
            print('Init package1.module1')
            from . import module2
        module2.py
            print('Init package1.module2')
            import sys, pprint
            pprint.pprint(sys.modules)
            from . import module1

I get:

vic@ubuntu:~/Desktop/app2$ python3 start.py 
Init package1
Init package1.module1
Init package1.module2
{'__main__': <module '__main__' from 'start.py'>,
 ...
 'package1': <module 'package1' from '/home/vic/Desktop/app2/package1/__init__.py'>,
 'package1.module1': <module 'package1.module1' from '/home/vic/Desktop/app2/package1/module1.py'>,
 'package1.module2': <module 'package1.module2' from '/home/vic/Desktop/app2/package1/module2.py'>,
 ...
Traceback (most recent call last):
  File "start.py", line 3, in <module>
    from package1 import module1
  File "/home/vic/Desktop/app2/package1/module1.py", line 3, in <module>
    from . import module2
  File "/home/vic/Desktop/app2/package1/module2.py", line 5, in <module>
    from . import module1
ImportError: cannot import name module1
vic@ubuntu:~/Desktop/app2$ 

import package1.module1 works, but i want to use from . import module1 because i want to make package1 portable for my other applications, that's why i want to use relative paths.

I am using python 3.

I need circular imports. A function in module1 asserts that one of its parameter is instance of a class defined in module2 and viceversa.

In other words:

sys.modules contains 'package1.module1': <module 'package1.module1' from '/home/vic/Desktop/app2/package1/module1.py'>. I want to get a reference to it in form from . import module1, but it tries to get a name, not a package like in case import package1.module1 (which works fine). I tried import .module1 as m1 - but that's a syntax error.

Also, from . import module2 in module1 works fine, but from . import module1 in module2 doesn't work...

UPDATE:

This hack works (but i am looking for the 'official' way):

print('Init package1.module2')

import sys, pprint
pprint.pprint(sys.modules)

#from . import module1
parent_module_name = __name__.rpartition('.')[0]
module1 = sys.modules[parent_module_name + '.module1']

Answer 1

Circular imports should be generally avoided, see also this answer to a related question, or this article on effbot.org.

In this case the problem is that you import from . where . is the current package. So all your from . import X imports go through the package’s __init__.py.

You can make your problem a bit more visible, if you explicitely import your modules in the __init__.py and give them another name (and adjust the other imports to use those names of course):

print('Init package1')
from . import module1 as m1
from . import module2 as m2

Now when you are importing m1 in start.py, the package first initializes m1 and comes to the from . import m2 line. At that point, there is no m2 known in __init__.py so you get an import error. If you switch the import statements in __init__.py around (so you load m2 first), then in m2 it finds the from . import m1 line, which fails for the same reason as before.

If you don’t explicitely import the modules in __init__.py something similar still happens in the background. The difference is that you get a less flat structure (as the imports are no longer started from the package only). As such both module1 and module2 get “started” and you get the respective initialization prints.

To make it work, you could do an absolute import in module2. That way you could avoid that the package needs to resolve everything first, and make it reuse the import from start.py (as it has the same import path).

Or even better, you get rid of the circular import at all. It’s generally a sign that your application structure is not so good, if you have circular references.

(I hope my explanation makes any sense, I already had difficulties writing it, but the general idea should be clear, I hope…)

edit

In response to your update; what you are doing there is that you use the full package name to get the reference to the module. This is equivalent (but much more complicated) to the first possible option to make it work; you use an absolute import using the same import path as in start.py.

Answer 2

A better solution for your problem is to put package1 in it's own separate package. Of course then it can't import package2, but then again if it is reusable, why would it?

Answer 3

Your update emulates what the absolute import does: import package1.module1 if you do it while module1 being imported. If you'd like to use a dynamic parent package name then to import module1 in the module2.py:

import importlib
module1 = importlib.import_module('.module1', __package__)

---

I need circular imports. A function in module1 asserts that one of its parameter is instance of a class defined in module2 and viceversa.

You could move one the classes to a separate module to resolve the circular dependency or make the import at a function level if you don't want to use absolute imports.

.
├── start.py
#       from package1 import module1
└── package1
    ├── __init__.py
#           print("Init package1")
#           from . import module1, module2
    ├── c1.py
#           print("Init package1.c1")
#           class C1:
#               pass
    ├── module1.py
#           print("Init package1.module1")
#           from .c1 import C1
#           from .module2 import C2
    └── module2.py
#           print("Init package1.module2")
#           from .c1 import C1
#           class C2:
#               pass
#           def f():
#               from .module1 import C1

Output

Init package1
Init package1.module1
Init package1.c1
Init package1.module2

Another option that might be simpler than refactoring out c1.py is to merge module{1,2}.py into a single common.py. module{1,2}.py make the imports from common in this case.

Answer 4

module2.py
          import module1

Works too.

Answer 5

I ran into this same issue today, and it seems this is indeed broken in python3.4, but works in python3.5.

The changelog has an entry:

Circular imports involving relative imports are now supported. (Contributed by Brett Cannon and Antoine Pitrou in bpo-17636).

Looking through the bugreport, it seems that this not so much a buf fixed, as well as a new feature in the way imports work. Referring to poke's answer above, he shows that from . import foo means to load __init__.py and get foo from it (possibly from the implicitly loaded list of submodules). Since python3.5, from . import foo will do the same, but if foo is not available as an attribute, it will fall back to looking through the lists of loaded modules (sys.modules) to see if it is already present there, which fixes this particular case. I'm not 100% sure I properly presented how this works, though.

Answer 6

Make sure your package1 is a folder. Create a class in __init__.py -- say class1. Include your logic in a method under class1 -- say method1.

Now, write the following code -

from .package1 import class1
class1.method1()

This was my way of resolving it. To summarize, your root directory is . so write your import statement using . notations, e.g. from .package or from app.package.

Answer 7

The accepted answer to Circular import dependency in Python makes a good point:

If a depends on c and c depends on a, aren't they actually the same unit then?

You should really examine why you have split a and c into two packages, because either you have some code you should split off into another package (to make them both depend on that new package, but not each other), or you should merge them into one package.
— Lasse V. Karlsen♦

Maybe you should consider placing them in the same module. :)