0

I have a bug with imports in python I’m having a hard time to solve or even understand why it happens.I didn’t write the codebase so please forgive the fact that absolutely zero python naming conventions are followed. Actually my job is to refactor most of it and make it more pythonic.

For now a lot of files rely on manual path appending for importing other files. Like:

sys.path.append(os.path.join(os.path.dirname(__file__), “../Generic”))
from Generic import *

And they do that for all their imports from files that are in the project root or it's subfolders. This is obviously not ideal. I mean there are a dozen lines of those sys.path.append before the import section of those file.

My understanding is that this can be resolved by making the folders into python modules, just adding empty __init__.py files to each folder and then I should be able to do imports by naming the folders/subfolders. So I added those init files everywhere, I now have the following structure:

Deco_tool/
├─ Interface/
│  ├─ __init__.py
│  ├─ App1/
│  │  ├─ App1.py
│  │  ├─ __init__.py
│  ├─ App2/
│  ├─ App3/
│  ├─ Generic/
│  │  ├─ Generic.py
│  │  ├─ __init__.py
├─ Backend/
├─ __init__.py
├─ requirements.txt
├─ .gitignore

The Interface folder contains 3 subfolder for our 3 GUI applications and a Generic folder that contains a few classes that are used in the 3 GUI apps. App2 and App3 have the same structure as App1. For now you run one of the 3 GUI apps by executing the corresponding App1.py.

I want to import a class “Tabulation” defined in Generic.py in App1.py because it’s used there. So I replaced the sys.path.append by: from Interface.Generic.Generic import Tabulation

I get the following error: Traceback (most recent call last): File "\Interface\App1\App1.py", line 25, in from Interface.Generic.Generic import Tabulation ModuleNotFoundError: No module named 'Interface'

I don’t understand why. I tried a lot of combinations of module.submodule… paths, I tried relative import: from ..Generic import Tabulation (again I tried multiple variation of those, they all create the same error: ImportError: attempted relative import with no known parent package

Is it because the application gets launched from the App1.py that is way down in the subfolder structure ? It seems to be because if I just sys.path.append the whole project folder “Deco_tool/“ then it works fine. Is there another way to solve it that is more elegant than sys.path.append? I mean I'm already happy to only have one sys.path.append for the root folder instead of a dozen appends for each subfolder. But I would love to have none of those sys.path.append, because it's ugly, doesn't seem pythonic and also because the IDE (VSCode) doesn't follow them so marks all those classes and methods as undefined.

codeartha
  • 3
  • 2
  • Related: [Relative imports for the billionth time](https://stackoverflow.com/q/14132789/11082165). The answers there cover some common confusions about directories vs packages and scripts vs modules. – Brian61354270 Aug 10 '23 at 14:44
  • 2
    I'm very sorry you inherited this mess. It seems that the original developer(s) had no clue how Python's import system work. There is _absolutely no reason_ to use `sys.path` anywhere. Fortunately, this should be fixable with just a few minor reorganizations. – Brian61354270 Aug 10 '23 at 14:47
  • Concretely, these are some steps to get you started: 1) decide what you want the top-level package(s) to be. That's probably going to be either just `Deco_tools`, or both `Interface` and `Backend`. Which you pick is going to determine how all of your imports will look (e.g. `import Deco_tools.Interface.blah` vs `import Interface.blah`). 2) Put the directory _containing_ your top-level package(s) on the Python path. That usually happens in one of two ways: either install your project as a distribution package (preferable), or rely on this directory being your CWD. 3) you're done. – Brian61354270 Aug 10 '23 at 14:52
  • For installing your project as a distribution package, see [Packaging Python Projects](https://packaging.python.org/en/latest/tutorials/packaging-projects/). Basically, all you need to do is created a `pyproject.toml` file, pick a build system (e.g. poetry, hatch, flit, setuptools, ...), and optionally customize the config. Then, you can run `pip install .` to copy your project to your interpreter's site-packages, or `pip install -e .` to add your project's source directory to the current interpreter's Python path. Imports will work as expected, from any working directory. – Brian61354270 Aug 10 '23 at 14:56
  • For some build-system specific packaging guides, see [poetry - Basic usage](https://python-poetry.org/docs/basic-usage/), [hatch - Introduction](https://hatch.pypa.io/latest/intro/), [setuptools - Quickstart](https://setuptools.pypa.io/en/latest/userguide/quickstart.html) – Brian61354270 Aug 10 '23 at 15:06

0 Answers0