27

I'm working with a project that contains about 30 unique modules. It wasn't designed too well, so it's common that I create circular imports when adding some new functionality to the project.

Of course, when I add the circular import, I'm unaware of it. Sometimes it's pretty obvious I've made a circular import when I get an error like AttributeError: 'module' object has no attribute 'attribute' where I clearly defined 'attribute'. But other times, the code doesn't throw exceptions because of the way it's used.

So, to my question:

Is it possible to programmatically detect when and where a circular import is occuring?

The only solution I can think of so far is to have a module importTracking that contains a dict importingModules, a function importInProgress(file), which increments importingModules[file], and throws an error if it's greater than 1, and a function importComplete(file) which decrements importingModules[file]. All other modules would look like:

import importTracking
importTracking.importInProgress(__file__)
#module code goes here.
importTracking.importComplete(__file__)

But that looks really nasty, there's got to be a better way to do it, right?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Ponkadoodle
  • 5,777
  • 5
  • 38
  • 62
  • How do you know you create a circular import? If you're unaware of it, what's the problem? Please be specific about the problems. – S.Lott Mar 09 '10 at 03:25

4 Answers4

10

To avoid having to alter every module, you could stick your import-tracking functionality in a import hook, or in a customized __import__ you could stick in the built-ins -- the latter, for once, might work better, because __import__ gets called even if the module getting imported is already in sys.modules, which is the case during circular imports.

For the implementation I'd simply use a set of the modules "in the process of being imported", something like (benjaoming edit: Inserting a working snippet derived from original):

beingimported = set()
originalimport = __import__
def newimport(modulename, *args, **kwargs):
    if modulename in beingimported:
        print "Importing in circles", modulename, args, kwargs
        print "    Import stack trace -> ", beingimported
        # sys.exit(1) # Normally exiting is a bad idea.
    beingimported.add(modulename)
    result = originalimport(modulename, *args, **kwargs)
    if modulename in beingimported:
        beingimported.remove(modulename)
    return result
import __builtin__
__builtin__.__import__ = newimport
benjaoming
  • 2,135
  • 1
  • 21
  • 29
Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • 1
    Thanks, but: This breaks the original program in the two cases: 1) `class Somethink(object): myimport = __import__ ... self.myimport(name)` (self object is in modulename and the real module name is in etc, like in django.conf)` 2) `try: import some_module \n except: some_module = None` (remains in the set and it report false positive next time) – hynekcer Apr 16 '12 at 07:51
  • If a package foo_package has a module "foo_package/a.py" which imports `from foo_package import b` than it is detected as a circular import of "foo_package" even if "foo_package/__init__.py" is an empty file. This is false. The important module names are in the ignored "etc etc" not in modulename in the case of "from import "! – hynekcer Apr 16 '12 at 08:44
  • what I see is _empty_ `modulename` (i.e., it is a string of length 0). – sds Sep 27 '16 at 18:04
3

Circular imports in Python are not like PHP includes.

Python imported modules are loaded the first time into an import "handler", and kept there for the duration of the process. This handler assigns names in the local namespace for whatever is imported from that module, for every subsequent import. A module is unique, and a reference to that module name will always point to the same loaded module, regardless of where it was imported.

So if you have a circular module import, the loading of each file will happen once, and then each module will have names relating to the other module created into its namespace.

There could of course be problems when referring to specific names within both modules (when the circular imports occur BEFORE the class/function definitions that are referenced in the imports of the opposite modules), but you'll get an error if that happens.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Tor Valamo
  • 33,261
  • 11
  • 73
  • 81
  • 2
    Hey. Since imports usually happen at the top of files, isn't the situation you describe in the last paragraph the usual case. So circular imports usually throw an exception, *if* you can be sure you've actually imported one of the participating modules. Or have I misunderstood? – Jonathan Hartley Jul 04 '11 at 07:39
1

Not all circular imports are a problem, as you've found when an exception is not thrown.

When they are a problem, you'll get an exception the next time you try to run any of your tests. You can change the code when this happens.

I don't see any change required from this situation.

Example of when it's not a problem:

a.py

import b
a = 42
def f():
  return b.b

b.py

import a
b = 42
def f():
  return a.a
  • Well, as I said, it was a poorly designed project. I have a module which holds global settings for the project. I have other modules which configure preset settings. It's *possible* that those modules trigger a circular import, and don't through any errors, but just don't do their job. – Ponkadoodle Mar 09 '10 at 01:28
  • Circular imports will kill your ability to build docs using sphinx and autodoc. – dsz Jan 20 '23 at 08:18
0

import uses __builtin__.__import__(), so if you monkeypatch that then every import everywhere will pick up the changes. Note that a circular import is not necessarily a problem though.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • 1
    Circular imports will kill your ability to build docs using sphinx and autodoc. – dsz Jan 20 '23 at 08:17