Python moving modules into sub directory (without breaking existing import structure)

Question

Supposing I am writing library code with a directory structure as follows:

- mylibrary/
|
|-----foo.py
|-----bar.py
|-----baz.py
|-----__init__.py

And to better organise I create a sub directory:

- mylibrary/
|
|-----foobar/
|     |-----foo.py
|     |-----bar.py
|-----baz.py
|-----__init__.py

I want all client code to keep working without updates so I want to update init.py so that imports don't break.

I've tried adding this to init.py:

from foobar import foo

Now if I open a shell I can do:

from mylibrary import foo
print(foo.Foo)

However if I do this:

from mylibrary.foo import Foo

I get No module named mylibrary.foo error. Here is the traceback from my actual example:

Type "help", "copyright", "credits" or "license" for more information.
>>> from global_toolkit import section
>>> section.Section
<class 'global_toolkit.serialization.section.Section'>
>>> from global_toolkit.section import Section
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'global_toolkit.section'
>>>

Can anyone explain this behaviour?

I'm not sure I understand correctly, but why don't you just create another `__init__.py` inside `foobar/` directory? — Péter Leéh, Aug 15 '20 at 11:04
You need to add an `__init__.py` to the `foobar` directory. Then you can do `from mylibrary.foobar.foo import Foo`. There's no need to add anything to any of the `__init__.py` files. — ekhumoro, Aug 15 '20 at 11:05
I was unable to recreate this error, but as the others have pointed out, an `__init__.py` file may solve the problem. — The Otterlord, Aug 15 '20 at 11:06
@ekhumoro I can't do from mylibrary.foobar.foo import Foo because the client code would then break. I'm trying to change the layout of the project without breaking client imports. — Neil, Aug 15 '20 at 11:10
I'm also on Python 3.7 so just adding __init__ somewhere shouldn't make a difference — Neil, Aug 15 '20 at 11:12
@Neil We don't know what are the client imports. Just try to put an empty `__init__.py` inside `foobar/` and check if it works or not. — Péter Leéh, Aug 15 '20 at 11:16
@PéterLeéh okay I just tried this. Unfortunately didn't work. Note: I don't know what are the client imports either! I'm just trying to ensure that all imports that worked before the migration still work now, without modification. Perhaps that is not possible though. — Neil, Aug 15 '20 at 11:23
@Neil The problem is that `from foobar import foo` just adds a name to a namespace. You cannot import from a mere name, even if it happens to point to a module object (i.e. it fails for the same reason that `import os as x; from x import path` fails). — ekhumoro, Aug 15 '20 at 12:35

score 2 · Accepted Answer · answered Aug 15 '20 at 13:06

2

Add this in your __init__.py :

from .foobar import foo, bar
import sys
for i in ['foo','bar']:
  sys.modules['mylib.'+i] = sys.modules['mylib.foobar.'+i]

Now, from mylib.foo import Foo should work.

answered Aug 15 '20 at 13:06

Partha Mandal

1,391
8
14

Python moving modules into sub directory (without breaking existing import structure)

1 Answers1