1

I want to understand the root of my problem. I have encountered this a couple times and it is quite time consuming to figure it out each time.

This time I feel like I have missed something that I thought I understood. A related question is posted here: Infinite loop with Python imports; looking for Pythonic way

I decided my modules present an infinite loop, but I still get the same error. so here are what I have. I have included __init__.py files here too because I suspect that this file might cause behavior that I am currently unaware of.

In sources/preparation/features/__Init__.py:

from .build_features import *
from .get_qualified_edges import *
from .select_strategy import *
from .test import *

In sources/preprocessing/__init__.py, I have:

from .apply_preprocessing import *
from .convertion import *

In sources/preprocessing/apply_preprocessing I have:

from Sources.Preparation.Features.get_qualified_edges import get_all_GeneDisease_unweighted_disease2disease_qualified_edges
from Sources.Preparation.Features.get_qualified_edges import get_GPSim_disease2disease_qualified_edgesjk

In directory source/preparation/features/get_qualified_edges.py:

from Sources.Preprocessing.convertion import Converter # added this lien causes error to be raised
from itertools import combinations

In Sources/preprocessing/conversion.py I don't have anything imported.

Below are the sequence of files that are run because error arise:

sources\__init__.py
sources\preparation\__init__.py
sources\prepartion\features\__init__.py
sources\preparation\features\build_features.py
sources\preparation\features\get_qualified_edgdes.py
sources\preprocessing\__init__.py
sources\preprocessing\apply_preprocessing.py
\\error raise

The error is raised when I import a class from Sources.Preprocessing.convertion as shown below. In source/preparation/features/get_qualified_edges.py:

from Sources.Preprocessing.convertion import Converter # added this lien causes error to be raised

Let me know if you folks need more information about this problem.

What I want to know is: Why does this problem occur? I solved the problem when I moved Converter from Sources.Preprocessing.convertion to Sources.Preparation.Data.conversion.

Observation

What I observed is that the error disappeared when there are no "cross" imports between Sources.Preprocessing.modules_A (imported in Sources.Preparation.modules_C) and Sources.Preparation.modules_B (imported in Sources.Preprocessing.modules_D).

That's it. There are no direct "cross" imports between modules and instead there are "cross" imports between parent modules, if that makes any sense.

Community
  • 1
  • 1
A-nak Wannapaschaiyong
  • 198
  • 1
  • 2
  • 16
  • Your import structure is complicated so it's hard to be sure but I suspect you have module A that imports B, and B (directly or indirectly) imports A. And A defines a function that B needs *after* its `import B` statement. Imports in Python are executable statements, so the interpreter pauses processing A when told to import B. That may mean that a needed `def` in A hasn't yet been processed when B is imported. – BoarGules Feb 03 '20 at 01:10

1 Answers1

1

The problem lies in you are introducing cyclic imports from your files.

For files witin sources/preprocessing, you are importing things from source/preparation. But those files, in-turn, import things from source/preprocessing.

Once you have import chains that goes something like

A.py -> B.py -> C.py -> B.py -> E.py

(direct cycle)

or

A.py -> B.py -> C.py -> D.py-> B.py -> E.py

(indirect cycle)

then it won't be able to continue importing the modules since it is impossible for it to terminate.

A simple way to over come this in python is using local import.

For example, instead of

from Sources.Preprocessing.convertion import Converter 
# more code.....

class A:

  def foo():
    bar = Converter()

you can instead use local imports like

# more code.....

class A:

  def foo():
    from Sources.Preprocessing.convertion import Converter 

    bar = Converter()

such that it only imports before you are actually using the object. This will avoids importing everything at the start, which avoid cyclic imports.

Edit:

In short, to answer OP's question of this particular issue:

Somewhere in the code you initiated the import of preprocessing module. (probably in sources\preparation\features\get_qualified_edgdes.py? not too sure) At this time, preprocessing is added to sys.modules namespace but as an empty module (place holder).

Later on in the same import chain, in the file source/preparation/features/get_qualified_edges.py, the line 

from Sources.Preprocessing.convertion import Converter

tries to import convertion from Preprocessing module, but it will find an empty module (since it's still in the process of importing it) and hence failed to find the corresponding name (convertion in this case).

Tin Lai
  • 440
  • 3
  • 8
  • I understand your point, but how does this applies to my case? take a look at line that cause error when added in this way```from Sources.Preprocessing.convertion import Converter``` convertion.py does not import anything. So in my case, there is no any of direct or indirect cycle as you state. import just ended at convertion.py. I believe my question demands slightly different solution. – A-nak Wannapaschaiyong Feb 03 '20 at 05:02
  • I would think that `from Sources.Preparation.Features.get_qualified_edges import ...` from `preprocessing/apply_preprocessing.py` is the main cause of your import problem. Looking at the snipets, **preparation/features/__Init__.py** 's `from .get_qualified_edges import *` eventually creates an import chains leads to **preprocessing/apply_preprocessing** 's `from Sources.Preparation.Features.get_qualified_edges import ...` – Tin Lai Feb 03 '20 at 06:10
  • You should either refactor `preparation\features\get_qualified_edgdes.py` or use the proposed method of local imports. As for the reason of it only throwing errors with the additional `from Sources.Preprocessing.convertion import Converter`, it's because modern python will allow cyclic imports, where when it happens it will make it points to an empty module. But during the import of that module, if you refers to functions or class within it, it sees that it's an empty module (as it hasn't finish importing yet) and throw the exception of `ImportError` of not finding the referred name. – Tin Lai Feb 03 '20 at 06:28
  • Can you show what import chain that I create looks like? I just don't see why there are circular import in my cases since all of the imported file (other than __init__.py) import other file directly. I can see the same file get imported twice, but no circular import. Please show me what it looks like. – A-nak Wannapaschaiyong Feb 03 '20 at 23:32
  • You'd have to include more of your files content as your current description is insufficient where some files within the chain are missing. – Tin Lai Feb 04 '20 at 08:14