Test an examples directory for a Python project

Question

I have a project for developing a Python package where the structure of the project is similar to the following:

myproject/
├── README.md
├── examples/
│   ├── ex1.py
│   └── ex2.py
├── pyproject.toml
├── src/
│    └── mypackage/
│        ├── __init__.py
│        ├── adder.py
│        └── divider.py
└── tests/
    ├── test_adder.py
    ├── test_divider.py
    └── test_examples.py

The project is for developing a Python package named mypackage which is located in the src directory. The package is uploaded to PyPI where users can pip install it. Tests for the package are run with pytest and are located in the tests directory. Examples of using the package are in the examples directory. The examples are just scripts as shown below for ex1.py

"""
Example 1
"""
from mypackage import adder

x = 2.5
y = 8
a = adder(x, y)

print('a is', a)

The purpose of test_examples.py is to test the example files, its contents are shown below:

from examples import ex1
from examples import ex2

def test_ex1():
    ex1

def test_ex2():
    ex2

When I run pytest in the myproject directory I get the error shown here:

$ cd myproject
$ pytest

platform darwin -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/gavinw/Desktop/test-examples
collected 2 items / 1 error

================================================================== ERRORS ==================================================================
_________________________________________________ ERROR collecting tests/test_examples.py __________________________________________________
ImportError while importing test module '/Users/gavinw/Desktop/test-examples/tests/test_examples.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/miniconda3/envs/ztest/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_examples.py:1: in <module>
    from examples import ex1
E   ModuleNotFoundError: No module named 'examples'
========================================================= short test summary info ==========================================================
ERROR tests/test_examples.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================= 1 error in 0.05s =============================================================

It looks like pytest is not able to run the files in the examples directory because the location does not allow them to be imported. Any suggestions on how I can test the example files? Should I even use pytest for testing examples or is there a different testing tool for this?

I think that you should put the examples under the tests directory, so they can be imported properly. — Peter K, Oct 17 '22 at 05:08
@PeterK The purpose of the examples is to demonstrate how to use the package. I would like to keep them separate from the tests. — wigging, Oct 17 '22 at 13:10

score 1 · Accepted Answer · answered Oct 20 '22 at 03:05

There are two approaches that fix the problem. The first approach is to run pytest using the following command:

python -m pytest

The second approach is to add the project directory in the pyproject.toml file using the pythonpath setting as shown below. Then just use the pytest command to run the tests.

[tool.pytest.ini_options]
pythonpath = ["."]

score 0 · Answer 2 · answered Oct 17 '22 at 14:29

0

You should ensure that the examples directory contains __init__.py so it can be imported correctly.

If that is not enough, you can use PYTHONPATH:

PYTHONPATH="/path/to/your/code/project/:/path/to/your/code/project/examples" pytest

PYTHONPATH might be tricky, see https://stackoverflow.com/a/4580120/3800552 and https://stackoverflow.com/a/39682723/3800552 for some usage examples.

answered Oct 17 '22 at 14:29

Peter K

1,959
1
16
22

Why would I set the PYTHONPATH like this instead of using the pythonpath in the pytest configuration settings? Documentation for this is found [here](https://docs.pytest.org/en/7.1.x/reference/reference.html?highlight=pythonpath#confval-pythonpath). – wigging Oct 18 '22 at 13:45
No particular reason. My main bet is the missing `__init__.py` file - that should do the trick for you. – Peter K Oct 18 '22 at 15:11
If I run `python -m pytest` then everything works fine. There's no need to add an `__init__.py` in the examples directory. – wigging Oct 18 '22 at 17:10

Test an examples directory for a Python project

2 Answers2