python workaround for circular import

Question

Ok so it is like this.

I'd rather not give away my code but if you really need it I will. I have two modules that need a bit from each other. the modules are called webhandler and datahandler.

In webhandler I have a line:

import datahandler

and in datahandler I have another line:

import webhandler

Now I know this is terrible code and a circular import like this causes the code to run twice (which is what im trying to avoid).

However the datahandler module needs to access several functions from the webhandler module, and the webhandler module needs access to several variables that are generated in the datahandler module. I dont see any workaround other than moving functions to different modules but that would ruin the organisation of my program and make no logical sense with the module naming.

Any help?

don't import the entire module, just import what you need from it. ```from datahandler import myfunc``` and ```from webhandler import domthing``` — Lawrence Dean Vanderpool, Jul 13 '15 at 19:44
Still doesnt like it, it either throws errors or still runs the code twice — Adam Griffiths, Jul 13 '15 at 19:52
`from X import Y` doesn't help you at all. All the code in `X` will still run. — jez, Jul 13 '15 at 19:53

score 10 · Accepted Answer · answered Jul 13 '15 at 19:51

Circular dependencies are a form of code smell. If you have two modules that depend on each other, then that’s a very bad sign, and you should restructure your code.

There are a few different ways to do this; which one is best depends on what you are doing, and what parts of each module are actually used by another.

A very simple solution would be to just merge both modules, so you only have a single module that only depends on itself, or rather on its own contents. This is simple, but since you had separated modules before, it’s likely that you are introducing new problems that way because you no longer have a separation of concerns.
Another solution would be to make sure that the dependencies are actually required. If there are only a few parts of a module that depend on the other, maybe you could move those bits around in a way that the circular dependency is no longer required, or utilize the way imports work to make the circular dependencies no longer a problem.
The better solution would probably be to move the dependencies into a separate new module. If naming is really the hardest problem about that, then you’re probably doing it right. It might “ruin the organisation of [your] program” but since you have circular dependencies, there is something inherently wrong with your setup anyway.

I definitely don't agree that this is a form of "code smell". It's common to have definitions that are used in two different places in a "circular" fashion in many programming languages (C,C++, java, etc..), and the languages are designed to deal with it. This seems to be a shortcoming of python itself. — ldog, Jun 07 '23 at 07:28
For those like me who come across this answer when dealing with circular imports due to type-hinting, please note this is not a code smell! Please see other answers that correctly deal with the shortcoming of python, for example here: https://stackoverflow.com/a/69042351/177931 and here: https://stackoverflow.com/a/67673741/177931 — ldog, Jun 07 '23 at 07:42

Michael Scott Asato Cuthbert · Answer 2 · 2022-04-10T21:25:31.330

What others have said about not doing circular imports is the best solution, but if you end up absolutely needing them (possibly for backwards compatibility or clarity of code), it's usually within just one method or function of one of the modules. Thus you can safely do this:

# modA.py
import modB

# modB.py
def functionDependingOnA():
    import modA
    ...

There's a slight overhead to doing the import each time the function is called, but it is rather low unless it's called all the time. (about 400ns in my testing).

You could also do like this to avoid even that lookup:

# modA -- same as above.

# modB.py
_imports = {}

def _importA():
    import modA
    _imports['modA'] = modA
    return modA

def functionDependingOnA():
    modA = _imports.get('modA') or _importA()

This version only added 40ns of time on the second and subsequent calls, or about the same amount of time as an empty local function call.

On my system, the added time for the import after it's been imported once is about 380 nanoseconds, so if used in a microsecond or millisecond-running function context, you won't notice the import time. — Michael Scott Asato Cuthbert, Mar 14 '19 at 17:36

score 1 · Answer 3 · answered Oct 22 '19 at 13:16

SqlAlchemy uses the Dependency Injection pattern, where the required module is passed to a function by a decorator:

@util.dependencies("sqlalchemy.orm.util")
def identity_key(cls, orm_util, *args, **kwargs):
    return orm_util.identity_key(*args, **kwargs)

This approach is basically the same as doing an import inside a function, but has slightly better performance.

score 0 · Answer 4 · edited May 23 '17 at 11:54

the webhandler module needs access to several variables that are generated in the datahandler module

It might make sense to push any "generated" data to a third location. So datahandler functions call config.setvar( name, value ) when appropriate and webhandler functions call config.getvar( name ) when they need to. config would be a third sub-module, containing simple setvar and getvar functions that you write (wrappers around setting/getting elements of a global dictionary would be the simplest approach).

Then the datahandler code would import webhandler, config but webhandler would only need to import config.

I agree with poke however, that the need for such a question betrays the fact that you probably haven't yet got the design finalized as neatly and logically as you thought. If it were me, I would re-think the way modules are divided up.

python workaround for circular import

4 Answers4