2

I'm new at Python and previously I've been using languages like Swift, where import is not a big deal: you're just defining a new class and can access it from another part of your program.

I can't use this way with Python because here import works in the other way: you can't make cyclic imports, where two files import each other. I understand that I'm facing this problem because of using the language in a wrong way but I don't understand how to avoid it.

I mean, in most cases you just can solve this problem by combining two classes into a single file but it doesn't feel right. Also, I've found advice like "move your import statement to the end of file" but it doesn't feel like a good idea too.

I would like to understand the Python's philosophy, if you will. How should I organize my project and what should I be guided by, when deciding on making a class in a separate file?

cezar
  • 11,616
  • 6
  • 48
  • 84
Yury Imashev
  • 2,068
  • 1
  • 18
  • 32
  • 2
    What's the concrete reason why you (need to) have a cyclic relationship in the first place? – deceze May 15 '17 at 07:20
  • 2
    If there's something both modules need, put it in a separate module. – Peter Wood May 15 '17 at 07:22
  • Possible duplicate of [How to avoid circular imports in Python?](http://stackoverflow.com/questions/7336802/how-to-avoid-circular-imports-in-python) – Joseph Ireland May 15 '17 at 07:26
  • 1
    I don't know Swift, but in some languages like Java or PHP you have a file for each class. Python differs in this way, because here you have modules. A module can have many classes. You can and should write more then one class in a single file. Try to organize your code logically. – cezar May 15 '17 at 07:26
  • @deceze good question, actually. I'm facing this because I have a file `User.py` where `User` class (and some) enums are defined. And this class needs to extract some data from a database. So it imports `DataManager` class. But `DataManager` needs to know the `User` class to build it from the extracted data. – Yury Imashev May 15 '17 at 07:44
  • 2
    That sounds like the bigger issue… your `User` objects should probably only *hold* data, not actively know about how to *get* data too. – deceze May 15 '17 at 07:46
  • Check how python ORM libraries are designed. There're many to choose from: Django, SQLAlchemy, peewee and many others. No need to develop your own ORM framework especially when you just started learning Python. – Eugene Morozov May 15 '17 at 07:47
  • @deceze well, I agree and this specific case was already fixed. I'm just looking for some more abstract answer because the problem is my wrong approach. – Yury Imashev May 15 '17 at 07:51
  • @Yury, Can you give a more concrete example of the problem you're encountering, i.e., a minimal set of instructions that don't work? I made a guess in my answer below, but I'm not sure if I hit the mark. – Matthias Fripp May 15 '17 at 22:02

3 Answers3

10

You certainly can import child from parent and parent from child. The key to making this work is for child not to probe too deeply into parent from its module-level code, because the parent module is only partly loaded at the time that the Python runs the module-level code in child.

Here's what happens when you import child from parent and parent from child (assuming parent is loaded first):

  1. Code at the module level of parent runs until it reaches a statement that loads child (either import child or from child import something). By "module level", I mean the statements that aren't within a class or function definition. The classes and functions defined at the module level will also be created as objects within the module. However, the functions and class methods themselves will not be run yet.
  2. When Python gets to the import child statement (or equivalent) in parent's module-level code, it will stop running the parent code and begin running the module-level code in child. If child imports parent via import parent or from parent import something, it will get the parent module in its current, partially constructed state. So module-level code in child cannot access objects that are defined below import child in parent.py.
  3. Once child's module-level code finishes running, control will return to parent below the import child statement, and Python will finish running all of the module-level code in parent.

This process will give you trouble if child's module-level code tries to access objects that are defined in parent after the import child statement (since parent is only partially finished when child loads). The solution to this is to import parent at the module level of child but defer accessing objects within parent from child until after child and parent have finished loading. In particular, instead of using from parent import something in child's module-level code, you may need to use import parent, then access parent.something from inside function or method code in child. This is safe to do because those functions and methods won't be run until after child and parent finish running, at which point all the elements of the parent module are correctly defined.

Here's an example of what I mean, using the setup you described in a comment. If you give more information on the code that is giving you problems, I could tailor this more closely.

Version 1 (won't work)

__main__.py:

from user import User
u = User()

user.py:

from data_manager import DataManager
...
class User:
    def __init__(self, data_manager=None):
        if data_manager is None:
            data_manager = DataManager(user=self)
        self.data_manager = data_manager

data_manager.py:

# next line will fail because user.py has been partly loaded
# and user.User doesn't exist yet
from user import User
...
class DataManager:
    def __init__(self, user=None):
        ...
        if user is None:
            user = User(data_manager=self)
        self.user = user

Version 2 (will work)

__main__.py:

from user import User
u = User()

user.py:

import data_manager as dm
...
class User:
    def __init__(self, data_manager=None):
        if data_manager is None:
            data_manager = dm.DataManager(user=self)
        self.data_manager = data_manager

data_manager.py:

import user as user_module
...
# this defines a class which will eventually create an instance
# of user.User, but it won't try to do that until control returns
# to __main__.py, by which point the `user` and `data_manager`
# modules are fully defined
class DataManager:
    def __init__(self, user=None):
        ...
        if user is None:
            user = user_module.User(data_manager=self)
        self.user = user

Note that references in your classes' __init__ methods don't get resolved until the class is actually instantiated. i.e., the user = user_module.User(data_manager=self) line does something like this: "look in the current module for an object called user_module; look in that object for a User attribute; construct an object of that class". The important thing is that data_manager can safely import the user module early on (the module exists already, even though it is only partly constructed), but the code above won't actually look for anything inside the user module until a DataManager object is instantiated, by which time user.User will be properly defined.

Matthias Fripp
  • 17,670
  • 5
  • 28
  • 45
  • How is it possible this answer is not upvoted more? It's precise and helped me a lot! Thank you! – Günther Jena Feb 14 '20 at 17:14
  • Great answer! So is it generally better to avoid `from x import MyClass` and instead do `import x`? How about more nested modules like `from x.y.z import MyClass`? – Jeppe Mar 24 '22 at 16:27
  • It's really a question of timing. If module `w` Imports module `x` at the top, then `w` won't contain any objects when the module-level code in `x` runs. So if `x` has `import w` at the module level, it will succeed, but `from w import obj` won't. However even if `import w` works, you still can't access `w.obj` while module `x` is loading. This is usually OK because you often only access `w.obj` from a function defined in `x`, which gets called after `x` finishes loading, returns control to `w`, then `w` finishes loading. So everything in `w` is defined before it calls that function in `x`. – Matthias Fripp Mar 25 '22 at 18:11
  • To put it another way, if you access `w.obj` in a function, Python doesn't check whether `obj` exists in module `w` until it gets to that line, i.e. sometime after the modules are loaded, when that function gets called. But if you put `from w import obj` In `x`'s module-level code, then Python will look for `obj` in module `w` while it is loading module `x`. `obj` doesn't exist yet (since module `w` imported `x` before it finished loading itself), so this will fail. – Matthias Fripp Mar 25 '22 at 18:22
-1

I can't use this way with Python because here import works in the other way: you can't make cyclic imports, where two files import each other.

Yes you can!

Say we have two files in the same working directory:

#file1.py
import file2
x = 22
y = 'spam'
print(x)
print(y)

and

#file2.py
import file1
print("file1")

Notice what happens when we import file1.py

>>> import file1
file1
22
spam

file1.py imports file2.py and file2.py imports file1.py.

GIZ
  • 4,409
  • 1
  • 24
  • 43
-4

This problem happens because you're writing Python program like you would write Swift or Java program. Such approach never works well: every language is different and has different best practices. If you write unpythonic code, not only it looks ugly and undecipherable to other Python developers, you're struggling with the language instead of enjoying it.

Just structure your code like python developers structure it. Group related classes in one module. If there's a rare case where there's hard to avoid import cycles (almost never happens), import offending dependency in the function or method instead of global scope.

Eugene Morozov
  • 15,081
  • 3
  • 25
  • 32