Imports inside python packages

Question

I was facing import error (ImportError: cannot import name 'ClassB') in following code:

dir structure:

main.py
test_pkg/
    __init__.py
    a.py
    b.py

main.py:

from test_pkg import ClassA, ClassB

__init__.py:

from .a import ClassA
from .b import ClassB

a.py:

from test_pkg import ClassB
class ClassA:
    pass

b.py:

class ClassB:
    pass

in past i fixed it by quick 'experiment' by adding full name in import in a.py:

from test_pkg.b import ClassB
class ClassA:
    pass

I have read about import machinery and according to :

This name will be used in various phases of the import search, and it may be the dotted path to a submodule, e.g. foo.bar.baz. In this case, Python first tries to import foo, then foo.bar, and finally foo.bar.baz. link2doc

I was expecting it will fail again, because it will try to import test_pkg during test_pkg import, but it is working. My question is why?

Also 2 additional questions:

1. is it proper approach to have cross modules dependencies?
2. is it ok to have modules imported in package init.py?

---

My analysis:

Based on readings i recognized that high probably issue is that, because off

__init__.py:
from .a import ClassA
from .b import ClassB

ClassA and ClassB import is executed as part of test_pkg import, but then hits import statement in a.py:

a.py:
from test_pkg import ClassB
class ClassA:
    pass

and it fail because circular dependency occured.

But is working when is imported using:

from test_pkg.b import ClassB

and according to my understanding it shouldnt, because:

This name will be used in various phases of the import search, and it may be the dotted path to a submodule, e.g. foo.bar.baz. In this case, Python first tries to import foo, then foo.bar, and finally foo.bar.baz. If any of the intermediate imports fail, a ModuleNotFoundError is raised.

so i was expecting same behavior for both imports.

Looks like import with full path is not launching problematic test_pkg import process

from test_pkg.b import ClassB

Answer 1

Your file is named b.py, but you're trying to import B, not import b.

Depending on your platform (see PEP 235 for details), this may work, or it may not. If it doesn't work, the symptoms will be exactly what you're seeing: ImportError: cannot import name 'B'.

The fix is to from test_okg import b. Or, if you want the module to be named B, rename the file to B.py.

This actually has nothing to do with packages (except that the error message you get says cannot import name 'B' instead of No module named 'B', because in a failed from … import statement, Python can't tell whether you were failing to import a module from a package, or some global name from a module).

---

So, why does this work?

from test_pkg.b import B

I was expecting it will fail again, because it will try to import test_pkg during test_pkg import, but it is working. My question is why?

Because importing test_pkg isn't the problem in the first place; importing test_pkg.B is. And you solved that problem by importing test_pkg.b instead.

test_pkg.b is successfully found in test_pkg/b.py.

And then, test_pkg.b.B is found within that module, and imported into your module, because of course there's a class B: statement in b.py.

---

For your followup questions:

is it proper approach to have cross modules dependencies?

There's nothing wrong with cross-module dependencies as long as they aren't circular, and yours aren't.

It's perfectly fine for test_pkg.a to import test_pkg.b with an absolute import, like from test_pkg import b.

However, it's usually better to use a relative import, like from . import b (unless you need dual-version code that works the same on Python 2.x and 3.x). PEP 328 explains the reasons why relative imports are usually better for intra-package dependencies (and why it's only "usually" rather than "always").

is it ok to have modules imported in package __init__.py?

Yes. In fact, this is a pretty common idiom, used for multiple purposes.

For example, see asyncio.__init__.py, which imports all of the public exports from each of its submodules and re-exports them. There are a handful of rarely-used names in the submodules, which don't start with _ but aren't included in __all__, and if you want to use those you need to import the submodule explicitly. But everything you're likely to need in a typical program is included in __all__ and re-exported by the package, so you can just write, e.g., asyncio.Lock instead of asyncio.locks.Lock.

Answer 2

The code I would prefer to write is probably:

main.py:

from test_pkg import A, B

b.py:

class B:
    pass

a.py:

from .b import B

class A:
    pass

__init__.py:

from .a import A
from .b import B

main.py:

from test_pkg import A, B

1. The proper approach is to have cross module dependencies but not circular. You should figure out the hierarchy of your project and arrange your dependency graph in a DAG (directed acyclic graph).

2. what you put in package __init__.py will be what you can access thru the package. Also you can refer to this question for the use of __all__ in __init__.py.