82

I have a Bottle webserver module with the following line:

from foobar.formtools import auto_process_form_insert

And the foobar.formtools module contains this line:

from foobar.webserver import redirect, redirect_back

Of course, both result in the following errors (respectively):

ImportError: cannot import name auto_process_form_insert
ImportError: cannot import name redirect

Is it simply a fact that in Python two modules can't import each other and all module imports must be hierarchical in nature, or am I doing something wrong? Alternatively, is there a workaround short of placing all these nice functions in new modules?

Hubro
  • 56,214
  • 69
  • 228
  • 381
  • 3
    As explained in my answer, it is possible for modules to import each other, but if you need to do that, you may want to reconsider your design. – icktoofay Jul 28 '12 at 05:50
  • 2
    I think this explanation much easier to understand: http://stackoverflow.com/questions/9642451/python-files-import-from-each-other – rizkiaditya24 Apr 13 '17 at 11:25
  • 2
    See also https://stackoverflow.com/questions/744373/circular-or-cyclic-imports-in-python – flow2k May 11 '18 at 21:38
  • While two modules can import each other, it can get messy in practice. I found that when some of my methods were decorated with Keras registration decorators, I would get double-registration errors unless I hid them as inner methods within another method of the class. It seemed one of the modules was being imported twice, despite the theory discussed here. – Josiah Yoder Aug 19 '22 at 16:21

2 Answers2

104

Modules can import each other cyclically, but there's a catch. In the simple case, it should work by moving the import statements to the bottom of the file or not using the from syntax.

Here's why that works:

When you import a module, Python first checks sys.modules. If it's in there, it just imports from there. If it's not there, it tries to import it in the normal way; basically, it finds the file and runs the stuff in it.

Running a module populates the module's contents. For example, say we have this module, creatively named example_opener:

import webbrowser

def open_example():
    webbrowser.open('http://www.example.com/')

At the start, the module is empty. Then Python executes:

import webbrowser

After that, the module only contains webbrowser. Then Python executes this:

def open_example():
    webbrowser.open('http://www.example.com/')

Python creates open_example. Now the module contains webbrowser and open_example.

Say webbrowser contained this code:

from example_opener import open_example

def open(url):
    print url

Say example_opener is imported first. This code is executed:

import webbrowser

webbrowser has not yet been imported, so Python executes the contents of webbrowser:

from example_opener import open_example

example_opener has been imported, but not yet fully executed. Python doesn't care. Python pulls the module out of sys.modules. At this point, example_opener is still empty. It hasn't defined open_example yet, nor even completed importing webbrowser. Python can't find open_example in example_opener, so it fails.

What if we imported open_example from the end of webbrowser and webbrowser from the end of example_opener? Python would start by executing this code:

def open_example():
    webbrowser.open('http://www.example.com/')

webbrowser does not exist yet, but it doesn't matter until open_example is called. Now example_opener contains only open_example. It then executes:

import webbrowser

It has not been imported yet, so Python executes webbrowser. It starts:

def open(url):
    print url

It defines open. Then it executes:

from example_opener import open_example

example_opener is in sys.modules, so it uses that. example_opener contains open_example, so it succeeds. Python finishes importing webbrowser. That concludes importing webbrowser from example_opener. That's the last thing in example_opener, so the import of example_opener finishes, successful, as well.

Hubro
  • 56,214
  • 69
  • 228
  • 381
icktoofay
  • 126,289
  • 21
  • 250
  • 231
  • 5
    Nice breakdown. This is usually what I have to do in django where sometimes you just have to import circularly. Import at the bottom or within the function or method scope. – jdi Jul 28 '12 at 05:39
  • @Codemonkey: I noticed you added that you can avoid using the `from ... import ...` syntax, but I already have that in my answer at the top: "In the simple case, it should work by [...] not using the `from` syntax." – icktoofay Jul 28 '12 at 05:54
39

Don't do from ... import .... Just do import ... and reference its objects using the module name.

pyrospade
  • 7,870
  • 4
  • 36
  • 52
  • 1
    I don't think that will address a circular import issue though? – jdi Jul 28 '12 at 05:41
  • 6
    @jdi: It will. At the time of the import, the module will be empty, but since it's not trying to get at anything inside of it (yet), it doesn't matter. – icktoofay Jul 28 '12 at 05:43
  • Ah, true. Thats an approach instead of importing directly at the bottom – jdi Jul 28 '12 at 05:47
  • 9
    I'd say it's not particularly recommended to put imports at the botttom or at mixed points throughout. As a general rule of thumb. – David Jul 22 '16 at 08:24
  • 4
    This makes maintainability much easier too, imagine someone seeing imports on the bottom, thinking its silly to be there, moving them to the top and watching your code break. – Jaden Travnik Jun 01 '17 at 00:46
  • 1
    If you try to access the second module's attributes at the module scope of the first module imported, this could have the same error. For example, say I `import foo` from inside `bar.py`. And I `import bar` from inside `foo.py`. Say `foo.py` gets processed first. `foo -> import bar -> import foo`, where the final `import foo` would reference the partially instantiated `foo` module. Now if inside bar.py, at the module scope, I did something like `bar_constant = max(100, foo.foo_constant)`, it will fail, since the `foo` that bar.py can get a handle to has not instantiated `foo_constant` yet. – ely Feb 03 '18 at 16:53