1

I am dealing with a problem with circular imports in Python. As a result of tackling this problem, I have had to become more knowledgeable about packages/imports in Python.

Some resources that I used include:

https://docs.python.org/3/tutorial/modules.html

https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html

How to avoid circular imports in Python?

Circular (or cyclic) imports in Python

I know this topic has been discussed extensively, and I have done my best to become familiar with that discussion, but I have a minimum working example from our code base that I have yet to be able to resolve using the skills I learned from the sources that I cited above. I ask for any advice regarding this example.

The folder structure for the example is as follows:

Source/
    Package/
        __init__.py
        A/
            __init__.py
            a.py
        B/
            __init__.py
            b1.py
            b2.py
        C/
            __init__.py
            c1.py
            c2.py  

Every __init__.py is empty. The contents of each file is as follows:

a.py

from Package.B.b1 import class_b1

b1.py

from Package.B.b2 import method_b2

class class_b1(object):
    def __init__(self):
        pass

b2.py

from Package.C.c1 import class_c1

def method_b2():
    pass

c1.py

import Package.C.c2 as class_c2

class class_c1(object):
   def __init__(self):
        pass

c2.py

from Package.B.b1 import class_b1

class class_c2(object):
    def __init__(self):
        pass

The logic is as follows. I export PYTHONPATH=/path/to/Source to place Package in the sys path.

I then go to Package/A and execute python a.py.

a.py loads a class from b1.py which loads a method from b2.py which loads a class from c1.py which loads a class from c2.py which loads a class from b1.py (namely the same class that a.py loaded). This results in an ImportError

Note that all of this is being done in an activated python 3.6.5 virtual env in Ubuntu 16.04.

Here is a trace:

(eye_tracking) gsandh16@count:~/Documents/PythonImportTesting/MWE_refactored/Source/Package/A$ python a.py 
Traceback (most recent call last):
  File "a.py", line 1, in <module>
    from Package.B.b1 import class_b1
  File "/home/gsandh16/Documents/PythonImportTesting/MWE_refactored/Source/Package/B/b1.py", line 1, in <module>
    from Package.B.b2 import method_b2
  File "/home/gsandh16/Documents/PythonImportTesting/MWE_refactored/Source/Package/B/b2.py", line 1, in <module>
    from Package.C.c1 import class_c1
  File "/home/gsandh16/Documents/PythonImportTesting/MWE_refactored/Source/Package/C/c1.py", line 1, in <module>
    import Package.C.c2 as class_c2
  File "/home/gsandh16/Documents/PythonImportTesting/MWE_refactored/Source/Package/C/c2.py", line 1, in <module>
    from Package.B.b1 import class_b1
ImportError: cannot import name 'class_b1'

I would like to resolve the issue without refactoring code or sticking imports in method calls (i.e. when they actually use the imported package). I have tried various absolute imports, relative imports, appending to sys.path by getting the location of the current file. My gut tells me it has to do with what the sys.path is as the program goes.

Thank you for any help regarding this issue.

gsandhu
  • 489
  • 5
  • 13
  • 1
    Can you load `method_b2` right before you need to use it? – cwallenpoole Feb 26 '19 at 20:10
  • 1
    Who gave you this codebase to work with? This really just sounds like circular imports built into the system - unless there's some other definition of `B.b1` somewhere that `c2.py` is trying to reference, it's literally files importing each other in a circle, which causes an `ImportError`. Now, I understand you're obfuscating the names - it could be a coincidence that the name of `Package` is the same as the name of an external package that something is supposed to call, in which case the pythonpath might actually be the problem. – Green Cloak Guy Feb 26 '19 at 20:12
  • 1
    Reorganize your packages. If `b` relies on `c` and `c` on `b` then your structure is bad. In their relation one should have the role of the import and the other the role of the importer. – Klaus D. Feb 26 '19 at 20:14
  • Depending on how you're able to restructure, you can use relative imports (e.g. `from ..B.b1` instead of `from Package.B.b1`) but if you're sitting outside of the `B` and `C` modules unable to modify either of them, it seems like it's just a badly built system – Green Cloak Guy Feb 26 '19 at 20:15
  • Yes, I can change the code such that method_b2 is loaded before use.I agree it is a badly built system, however I would like to avoid having to refactorin this code base (time constraints). The name, i.e. "Package" does not interfere with any other packages. In C, I can always stick an #ifdef #endif and call it a day. I was wondering if there was such mechanisms in Python. – gsandhu Feb 26 '19 at 20:18
  • 1
    Instead of ``from X import Y`` you can just import module ``import X`` or ``from X import *``. In your example scenario replacing import statement in c2.py with ``from Package.B.b1 import *`` saves the day (because during import, it will be detected that module is already in the memory), but better solution is to reorganize your code to avoid such situations. – unlut Feb 26 '19 at 20:23
  • @unlut Thank you, your solution did fix the problem. I will refactor the code base to avoid this situation when I am able to. Thank you everyone for your help. – gsandhu Feb 27 '19 at 17:23

0 Answers0