"ImportError: cannot import name ..." - raised on "import from" but not on direct import

Question

What is the precise rule under which this exception is raised by Python 3 interpreter?

There are plenty of SO questions about that, with excellent answers, but I could not find one that gave a clear, general, and logically precise definition of the circumstances when this exception occurs.

The documentation doesn't seem to be clear either. It says:

exception ImportError

Raised when an import statement fails to find the module definition or when a from ... import fails to find a name that is to be imported.

But this seems inconsistent with the following example.

I meant to ask for a general definition rather than a specific case, but to clarify my concerns, here's an example:

# code/t.py:
from code import d

# code/d.py
from code import t

Running module t.py from the command line results in ImportError: cannot import name d.

On the other hand, the following code doesn't raise exceptions:

# code/t.py:
import code.d

# code/d.py
import code.t

At all times, __init__.py is empty.

In this example, the only modules or names mentioned in the import statement are t and d, and they were both clearly found. If the documentation implies that some name within the d module isn't found, it's certainly not obvious; and on top of that, I'd expect it to raise NameError: name ... is not defined exception rather than ImportError.

Answer 1

If abc is a package and xyz is a module, and if abc's __init__.py defines an __all__ that does not include xyz, then you won't be able to do from abc import xyz, but you'll still be able to do import abc.xyz.

Edit: The short answer is: your problem is that your imports are circular. Modules t and d try to import each other. This won't work. Don't do it. I'm going to explain the whole thing, below but the explanation is pretty long.

To understand why it gives an ImportError, try to follow the code execution. If you look at the full traceback instead of just the final part, you can see what it's doing. With your setup I get a traceback like this (I called the package "testpack" instead of "code"):

Traceback (most recent call last):
  File "t.py", line 1, in <module>
    from testpack import d
  File "C:\Documents and Settings\BrenBarn\My Documents\Python\testpack\d.py", line 1, in <module>
    from testpack import t
  File "C:\Documents and Settings\BrenBarn\My Documents\Python\testpack\t.py", line 1, in <module>
    from testpack import d
ImportError: cannot import name d

You can see what Python is doing here.

1. In loading t.py, the first thing it sees is from testpack import d.
2. At that point, Python executes the d.py file to load that module.
3. But the first thing it finds there is from testpack import t.
4. It already is loading t.py once, but t as the main script is different than t as a module, so it tries to load t.py again.
5. The first thing it sees is from testpack import d, which would mean it should try to load d.py . . . but it already was trying to load d.py back in step 2. Since trying to import d led back to trying to import d again, Python realizes it can't import d and throws ImportError.

Step 4 is kind of anomalous here because you ran a file in the package directly, which isn't the usual way to do things. See this question for an explanation of why importing a module is different from running it directly. If you try to import t instead (with from testpack import t), Python realizes the circularity one step sooner, and you get a simpler traceback:

>>> from testpack import t
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    from testpack import t
  File "C:\Documents and Settings\BrenBarn\My Documents\Python\testpack\t.py", line 1, in <module>
    from testpack import d
  File "C:\Documents and Settings\BrenBarn\My Documents\Python\testpack\d.py", line 1, in <module>
    from testpack import t
ImportError: cannot import name t

Notice that here the error is that it can't import t. It knows it can't, because when I told it to import t, it found itself looping back to import t again. In your original example, it didn't notice it was running t.py twice, because the first time was the main script and the second was an import, so it took one more step and tried to import d again.

Now, why doesn't this happen when you do import code.d? The answer is just because you don't actually try to use the imported modules In this case, it happens as follows (I'm going to explain as if you did from code import t rather than running it as a script):

1. It starts to import t. When it does this, it provisionally marks the module code.t as imported, even though it's not done importing yet.
2. It finds it has to do import code.d, so it runs d.
3. In d, it finds import code.t, but since code.t is already marked as imported, it doesn't try to import it again.
4. Since d finished without actually using t, it gets to go back and finish loading t. No problem.

The key difference is that the names t and d are not directly accessible to each other here; they are mediated by the package code, so Python doesn't actually have to finish "deciding what t is" until it is actually used. With from code import t, since the value has to be assigned to the variable t, Python has to know what it is right away.

You can see the problem, though if you make d.py look like this:

import code.t
print code.t

Now, after step 2, while running d, it actually tries to access the half-imported module t. This will raise an AttributeError because, since the module hasn't been fully imported yet, it hasn't been attached to the package code.

Note that it would be fine as long as the use of code.t didn't happen until after d finished running. This will work fine in d.py:

import code.t
def f():
    print code.t

You can call f later and it will work. The reason is that it doesn't need to use code.t until after d finished executing, and after d finishes executing, it can go back and finish executing t.

To reiterate, the main moral of the story is don't use circular imports. It leads to all kinds of headaches. Instead, factor out common code into a third module imported by both modules.

Answer 2

from abc import xyz

is equivalent to doing

xyz = __import__('abc').xyz

Since if you merely import abc, abc.xyz won't exist without a separate import (unless abc/__init__.py contains an explicit import for xyz), what you're seeing is expected behavior.

Answer 3

The problem is abc is a predefined standard library module and just creating a subdirectory of that same name with an __init__.py in it doesn't change that fact. Change the name of your package to something else by renaming the folder the __init__.py file is in to something different, i.e. to def, and then both forms of import should execute without error.