python pytest importing modules which import local modules

Question

I have the following python3 project structure:

tests/
    - testsuite_service1/
        - test_main.py
    - testsuite_service2/
        - test_main.py
src/
    - service1/
       - codebase/
           - __init__.py
           - main.py
           - logger.py
           - waiter.py
    - service2/
       - codebase/
           - __init__.py
           - main.py
           - logger.py
           - waiter.py

In my tests (pytest), I am importing like so:

from src.service1.codebase.waiter import check_status

In order to import a specific function within the service1 module.

Within waiter.py in service1 I am importing a function from logger.py like so

from logger import configure_logger

however, when running the tests, I get the error:

Traceback:
tests/test_main/test_main.py:3: in <module>
    from src.service1.codebase.waiter import check_status
src/codebase/waiter.py:8: in <module>
    from logger import configure_logger
E   ModuleNotFoundError: No module named 'logger'.

This occurs when the test runs, it feels like waiter.py is not looking in its local directory to find logger.py but rather may be in some other location?

Changing it to

from .logger import configure_logger

does solve the issue but causes issues with the runtime I plan on deploying this into (single zip of service1 with no parent directory) and thus gives error Unable to import module 'main': attempted relative import with no known parent package

Is it possible to not use the relative import and have my test understand where to find the desired files?

Have you tried `from src.service1.codebase.logger import configure_logger`? — b-fg, Nov 11 '18 at 12:04
Also looks like you are missing a `__init__.py` file in the root level of the package? — b-fg, Nov 11 '18 at 12:06
Possible duplicate of [Using pytest with a src layer](https://stackoverflow.com/questions/50155464/using-pytest-with-a-src-layer) — hoefling, Nov 11 '18 at 12:48

score 7 · Answer 1 · answered Nov 11 '18 at 12:40

7

Adding import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__))) to __init__.py in the codebase/ folder solved the issue.

answered Nov 11 '18 at 12:40

djsnakz

468
2
6
15

1

Thanks, this worked for me. Can you please explain why adding thing sys.path makes the imports correct? – Xavier Silva Sep 10 '20 at 11:28
1

@XavierSilva, it works by adding the package/directory name to your path. As a result, any of the modules within that package can be accessed and imported. – CodeBiker May 20 '21 at 18:50

score 1 · Answer 2 · answered Nov 11 '18 at 12:09

1

The package src should contain the __init__.py file at root level to indicate it is a package. Then you can use absolute or relative paths to import from other files. The absolute path would be

from src.service1.codebase.logger import configure_logger

But your relative path should work to after including the __init__.py file.

Check this answer for more info.

answered Nov 11 '18 at 12:09

b-fg

3,959
2
28
44

Thanks - However src won't exist in the finalized zip file. essentially, in the end, the only files that will actually be deployed is the /codebase folder contents within each service so it won't know anything about src (or even codebase folder) file path at deployment time this is why it needs to stay `import x from y` if possible - maybe I need to move my tests into their respective service folders? – djsnakz Nov 11 '18 at 12:25
You should have included this info in the answer... Anyways, this is likely to be a IDE configuration issue with the python path. – b-fg Nov 11 '18 at 12:37

python pytest importing modules which import local modules

2 Answers2