0

This problem has been eating away at me for a while. For some reason, the way I import my modules works only either for testing or for library usage. Both times the error is a Module not found error. Say I have the following Python repo:

- src
  - example
    - dir
      - internal.py
      - __init__.py
    - main.py
    - __init__.py
- test
  - test_main.py
  - __init__.py
- setup.cfg
- setup.py

Within setup.cfg, I have:

[metadata]
name = wf
version = 1.0.0

[options]
include_package_data = True
packages = find:

[options.packages.find]
where=src

I don't want the user to import from src, but instead import from example. After all, nobody imports from src, and this is the conventional structure, right?

However, now I've been encountering 2 types of errors, depending on my choice:

Choice 1

For python files in src and test, all imports are of the form from src.example.... import .... In this case, unit tests work without issues, in particular python -m unittest discover test. However, if I create a virtual environment in a separate directory, and pip install this directory and run

# script.py
from example.main import ...
# code that calls functions in main that import from internal.py

I encounter ModuleNotFoundError: no module named 'dir'. Only if I remove the src from imports do I avoid this issue.

Choice 2

I'm guessing the error is because of how I defined the setup in my cfg file, so I remove src from src.example.... import ... within the example directory. But now, my unit tests fail! This time, the error is ModuleNotFoundError: No module named 'dir'.

I've looked into some other options.

  • Relative imports: I've heard relative imports are fragile, so I avoid them in favor of absolute imports, which IDEs like PyCharm support very easily.
  • Sys add path: I've heard that people have complained about how this is a wordy solution, and it is not agnostic, and I agree since I plan to scale up testing, so I would like to avoid it too if possible. I'm referring to this post. I'm also aware of the 'nose' suggestion underneath, but do we really need a 3rd party for this?

This has to be a commonly encountered issue... if someone knows where the fix lies, or if there is some template example Python repository which addresses these concerns, I'd be very thankful!

wangf22
  • 1
  • 3
  • You will need to install the package before it may be imported normally across any working directory as `from example import ...`. Moreover, your `setup.py` does not contain the required code to actually setup the package. Please refer to [What is setup.py](https://stackoverflow.com/questions/1471994/what-is-setup-py) for a guide. – metatoaster Feb 14 '23 at 03:48
  • @metatoaster by working directory do you mean the test directory? – wangf22 Feb 14 '23 at 03:53
  • As in, no matter where what working directory the program is started from, e.g. can start from `/home/user`, `/`, `/tmp`, anywhere, with the import statement working as expected. – metatoaster Feb 14 '23 at 03:56
  • I see, I made some changes and it worked. I was under the misconception that the test directory didn't require a package install, and that the test directory had different properties than an ordinary working directory in that it was supposed to call the code from the project directly instead of from the package. Thanks. – wangf22 Feb 14 '23 at 03:59
  • There are situations where the scenario you have described (i.e. testing) will work, but this typically require manipulation of `PYTHONPATH` and thus non-standard. Just make sure your package is created/installed correctly in the active Python environment for a sane working state. – metatoaster Feb 14 '23 at 04:15

0 Answers0