Remove python circular import

Question

user.py:

from story import Story

class User:
    ...
    def get_stories(self):
        story_ids = [select from database]
        return [Story.get_by_id(id) for id in story_ids]

story.py

from user import User

class Story:
    ...
    def __init__(self, id, user_id, content):
        self.id = id
        self.user = User.get_by_id(user_id)
        self.content = content

as you can see, there is a circular import in this program, which causes an ImportError. I learned that I can move the import statement in method definition to prevent this error. But I still want to know, is there a way to remove circular import in this case, or, is it necessary (for a good design) ?

Answer 1

Another way to mitigate the circularity is to change the import style. Change from story import Story to import story, then refer to the class as story.Story. Since you only refer to the class inside a method, it won't need to access the class until the method is called, by which time the import will have completed successfully. (You may have to make this change in either or both modules, depending on which one is imported first.)

The design does seem somewhat strange, however. Your design is such that the User and Story classes are very tightly coupled -- neither can be used without the other. In such a case, it would usually make more sense to have them both in the same module.

Answer 2

The most obvious solution in this case is to break the dependency to the User class completely, by changing the interface so that the Story constructor accepts an actual User, not a user_id. This also leads to a more efficient design: for example if a user has many stories, the same object can be given to all of those constructors.

Other than that, the import of a whole module (that is story, and user instead of the members) should work - the module imported first will appear empty at the time when the second is imported; however it does not matter as the contents of these modules are not used at the global scope.

This is slightly preferable over importing within a method. Importing within a method has significant overhead over just a module-global lookup (story.Story), because it needs to be done for each method call; seems that in a simple case the overhead is at least 30-fold.

Answer 3

There are a bunch of these python circular import questions on the web. I chose to contribute to this thread because the query has a comment by Ray Hettinger that legitimizes the use case of a circular import, but recommends a solution that I believe is not especially good practice - moving the import to a method.

Apart from Hettinger's authority, three disclaimers to common objections are necessary:

1. I've never programmed in Java. I'm not trying to do Java style.
2. Refactoring is not always useful or effective. Logical API sometimes dictates a structure that makes recursive import references unavoidable. Remember, code exists for users, not programmers.
3. Combining modules that are quite large can cause readability and maintainability issues that may be much worse than one or two recursive imports.

In addition, I believe maintainability and readability dictates that imports be grouped at the top of the file, occur only once for each needed name, and that the from module import name style is preferable (except perhaps for very short module names with many functions, e.g. gtk), as it avoids repetitive verbal clutter and makes dependencies explicit.

With that out of the way, I will posit a simplified version of my own use case that brought me here, and provide my solution.

I have two modules, each defining many classes. surface defines geometric surfaces like planes, spheres, hyperboloids, etc. path defines planar geometric figures like lines, circles hyperbolae, etc. Logically, these are distinct categories and refactoring is not an option from the perspective of API requirements. Nevertheless, these two categories are intimate.

A useful operation is intersecting two surfaces, for example, the intersection of two planes is a line, or the intersection of a plane and a sphere is a circle.

If for example, in surface.py you do the straight forward import needed to implement the return value for an intersection operation:

from path import Line

you get:

Traceback (most recent call last):
  File "surface.py", line 62, in <module>
    from path import Line
  File ".../path.py", line 25, in <module>
    from surface import Plane
  File ".../surface.py", line 62, in <module>
    from path import Line
ImportError: cannot import name Line

Geometrically, planes are used to define the paths, after all, they may be arbitrarily oriented in three (or more) dimensions. The traceback tells you both what is happening and the solution.

Simply replace the import statement in surface.py with:

try: from path import Line
except ImportError: pass # skip circular import second pass

The sequence of operations in the trace back is still happening. It is just that the second time through, we ignore the import failure. This does not matter, since Line is not used at the module level. Therefore, the necessary namespace of surface is loaded into path. The namespace parsing of path can therefore complete, permitting it to be loaded into surface, completing the first encounter with from path import Line. Thus the namespace parsing of surface can proceed and complete, continuing on to whatever else may be necessary.

It is an easy and very clear idiom. The try: ... except ... syntax clearly and succinctly documents the circular import issue, easing whatever future maintenance may be required. Use it whenever a refactor really is a bad idea.

Answer 4

As BrenBarn said, the most obvious solution is to keep User and Story in the same module, which makes perfect sense if User is supposed to know anything about Story. Now if you really need to have them in distinct modules you can also monkeypatch User in story.py to add the get_stories method. It's a readability / decoupling trade off...