How to structure object-oriented Python 3 project and its imports?

Question

I have a object-oriented Python 3.7 project with a structure along the following lines:

├── plugins
│   ├── book_management
│   │   ├── book_inserter.py
│   │   ├── book_remover.py
│   │   ├── __init__.py
│   │   ├── book.py
│   │   ├── book_sampler.py
│   │   ├── operators
│   │   │   ├── __init__.py
│   │   │   ├── register_book.py
│   │   │   ├── unregister_book.py
│   │   │   └── mark_book_as_missing.py
│   ├── __init__.py
│   ├── reader_management
│   │   ├── __init__.py
│   │   ├── reader.py
│   │   ├── reader_creator.py
│   │   ├── reader_emailer.py
│   │   ├── reader_remover.py
│   │   ├── operators
│   │   │   ├── __init__.py
│   │   │   ├── create_reader.py
│   │   │   ├── remove_reader.py
│   │   │   └── email_reader.py
├── tests
│   ├── __init__.py
│   ├── book_management_tests
│   │   ├── __init__.py
│   │   ├── test_book.py
│   │   ├── test_book_inserter.py
│   │   ├── test_book_remover.py
│   │   ├── test_book_sampler.py
│   │   ├── test_mark_book_as_missing_operator.py
│   │   ├── test_register_book_operator.py
│   │   ├── test_unregister_book_operator.py
│   ├── reader_management_tests
│   │   ├── __init__.py
│   │   ├── test_reader.py
│   │   ├── test_reader_creator.py

In a test like test_mark_book_as_missing_operator I end up having imports like:

from plugins.book_management.book_inserter import BookInserter
from plugins.book_management.operators.mark_book_as_missing import (
    MarkBookAsMissingOperator
)
from plugins.reader_management.reader_creator import ReaderCreator
from plugins.reader_management.operators.create_reader import (
    CreateReaderOperator
)

This feels really bad having these really verbose partial imports. So I am guessing I must be doing it wrong. Ideally, importing plugins.reader_management and plugins.reader_management.operators possibly as something shorter would seem much more readable.

book_inserter.py is defining a single class BookInserter. Ideally, I would like to keep this 1-class / 1-file structure. Obvisouly, this leads to an inflation of the number of files but also allows of shorter more focused files. But if this is deeply non-Pythonic I am willing to hear why and how I should adapt the code structure.

Finally I have been using this kind of several layer architectures (plugins/*_management/operators/*.py) but this leads to very long import lines, and I am frequently having legitimate lint issues as a result.

I have been considering importing submodules from top modules (like book_management, in book_management/__init__.py) but I am not sure if it is good practice and also it seems like a violation of the principle of not having unused imports in your files. (Also would I be at risk of circular imports as a result?)

My main question in short: what would be a(the?) Pythonic way to structure such a project and setup the imports (with ideally some justification of why this would be a/the Pythonic way to do it).

https://stackoverflow.com/questions/35727134/module-imports-and-init-py-in-python — Cohan, Jan 17 '20 at 15:36

MisterMiyagi · Accepted Answer · 2020-01-17T16:16:47.567

It is perfectly fine to use __init__.py to compress your namespace. Use __all__ to clearly define that imported names are meant for export.

# plugins/book_management/__init__.py
from .book_inserter import BookInserter
from .operators.mark_book_as_missing import MarkBookAsMissingOperator
# more imports

__all__ = [
    'BookInserter',
    'MarkBookAsMissingOperator',
    # more exports
]

This reduces the length and number of imports on usage:

# test_mark_book_as_missing_operator
from plugins.book_management import BookInserter, MarkBookAsMissingOperator
from plugins.reader_management import ReaderCreator, CreateReaderOperator

There seems not to be a consensus whether 1-definition-per-file is a bad thing. For the standard library and many third-party modules it is customary to keep all directly related classes and functions together, though.

Is there a way to do it with absolute imports or would that lead to circular imports? — sunless, Jan 17 '20 at 15:42
@sunless Do you mean ``from plugins.book_inserter import BookInserter`` instead? Relative imports have exactly the same behaviour as such absolute imports, they merely avoid repeating and hard-coding the package name. — MisterMiyagi, Jan 17 '20 at 15:44

How to structure object-oriented Python 3 project and its imports?

1 Answers1