What is the correct folder structure to use for a Python project using pytest?

Question

I try to organized my Python projects using a folder structure. When I need to make tests I use something like the following.

.
|-- src
|   |-- b.py
|   `-- main.py
`-- tests
    `-- test_main.py

There is just one big problem with this approach. Pytest won't run if main.py is importing b.py.

So far I've tried placing empty __init__.py files inside the src and tests folders, both independently and together, but any of those seems to work.

It seems to me this is a pretty standard project, but I haven't been able to find a solution online. Should I use a different folder structure? Is there any recommended way to use pytest with this kind of projects?

---

This are the contents of the files:

# b.py
def triplicate(x):
        return x * 3

# main.py
from b import triplicate

def duplicate(x):
        return x * 2

# test_main.py
from src.main import duplicate


def test_duplicate():
        assert duplicate(2) == 4

And this is the error I get when running pytest:

==================================================================================================== ERRORS ====================================================================================================
_____________________________________________________________________________________ ERROR collecting tests/test_main.py ______________________________________________________________________________________
ImportError while importing test module 'C:\Users\edwar\test_pytest\tests\test_main.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
c:\python39\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests\test_main.py:1: in <module>
    from src.main import duplicate
src\main.py:1: in <module>
    from b import triplicate
E   ModuleNotFoundError: No module named 'b'
=========================================================================================== short test summary info ============================================================================================
ERROR tests/test_main.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================================================================== 1 error in 0.15s ===============================================================================================

Answer 1

Python uses the 'environment variable' PYTHONPATH to look for sources to import code from. By default, the directory you execute a python program is automatically included, but you want to include something like this when you test:

PYTHONPATH=$PYTHONPATH,../src python test_main.py

This is if you're executing a test from the source directory. Tools like IntelliJ (PyCharm) will let you add this as a value in your test invocation. Alternatively you can use export PYTHONPATH=.... (Note this is for a *nix environment, your mileage on windows may vary.)

The upshot is that every directory in PYTHONPATH will be loaded and Python will attempt to use it as a 'root' for modules you try to import. Your basic directory structure is the most idiomatic.

• See this answer for more on configuring PYTHONPATH correctly.
• See this doc for more about how the PYTHONPATH is modified and used 'under the hood'.
• See this answer for options to include the src directory when running pytest tests.
• See this blog post about using autoenv (a Python library) to enable the usage of .env files to manage this for you (at least within a virtualenv setup - a good idea generally).
• setup.py is also idiomatic for including many modules, and may provide a more convenient path for the situation you're handling.

Answer 2

Edit: After posting, I realized that someone has already posted this solution in a different question, many years ago.

If you just want to run all the tests from the root directory without having to worry about the PYTHONPATH, the easiest solution would be to simply invoke the tests by typing in the terminal:

python -m pytest

While just running pytest would fail in the OP example, python -m pytest would work, as according to the docs

... calling via python will also add the current directory to sys.path.

Hope that helps!

Answer 3

You can put your source files in the root directory and adjust the import paths as such:

main.py

# main.py
from stackoverflow.b import triplicate

def duplicate(x):
        return x * 2

b.py

# b.py
def triplicate(x):
        return x * 3

test_main.py

# test_main.py
from stackoverflow.main import duplicate


def test_duplicate():
        assert duplicate(2) == 4

Then running pytest . from the root directory stackoverflow:

collected 1 item

tests\test_main.py .                                                                                                                                                                      [100%] 

====================================================================================== 1 passed in 0.03s ======================================================================================= 

Or, if you wish to keep the src folder, your import would be this:

from stackoverflow.src.b import triplicate