pytest finding and not finding modules to test

Question

I was going to post this question in code review because rather than a solution I wanted for python experts to check my code. But while preparing the code I found some related issue so I finally decided to ask it here since it is not more just a code check

My question is how does pytest finds modules to test? Let me explain

Situation 1 First I have this code structure

|
|-tests
|    |----test_sum.py
|
|--script_to_test.py

test_sum.py

import script_to_test

def test_sum():
    d=script_to_test.sum(6,5)
    assert d==11

script_to_test.py

def sum(a,b):
    return a+b

If I create a virtual environment and install pytest there, or if I use a conda environment with pytest installed I can do

pytest

and I will get

pytest 
====================================================================== test session starts ======================================================================
platform linux -- Python 3.9.16, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/me/pytestExample
collected 1 item                                                                                                                                                

tests/test_sum.py .                                                                                                                                       [100%]

======================================================================= 1 passed in 0.01s =======================================================================

Situation 2 If I add a file pytest.ini in the root

[pytest]
python_files = test_*
python_classes = *Tests
python_functions = test_*

pytest will not work anymore and I will get

 pytest
========================================================= test session starts ==========================================================
platform linux -- Python 3.9.16, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/me/pytestExample, configfile: pytest.ini
collected 0 items / 1 error                                                                                                            

================================================================ ERRORS ================================================================
__________________________________________________ ERROR collecting tests/test_sum.py __________________________________________________
ImportError while importing test module '/home/me/pytestExample/tests/test_sum.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../.pyenv/versions/3.9.16/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_sum.py:1: in <module>
    import script_to_test
E   ModuleNotFoundError: No module named 'script_to_test'
======================================================= short test summary info ========================================================
ERROR tests/test_sum.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================== 1 error in 0.07s ===========================================================

Now, I can solve this already, but the purpose of this question is to know why it fails to work when a pytest.ini is added? and why does it work in the first place when that file is not present?

Answer 1

The default glob patterns for test filename discovery are:

["test_*.py", "*_test.py"]

This is documented here and you can see it in the pytest source here.

That means that pytest considers your file script_to_test.py as a test file itself, since it's matching the second pattern *_test.py. If you add a function named test_something() inside that script_to_test.py file, it will also be called during the test run.

During the test discovery process, pytest will import all the test modules, which makes them available in sys.modules. When your test actually runs, the import statement in test_sum.py is able to resolve the import because script_to_test has already been imported (it's cached in sys.modules). You could add this to the top of test_sum.py to verify:

import sys
if "script_to_test" in sys.modules:
    print("It's already imported!\n")

...

Run with pytest -s to show the print output.

collecting ...
It's already imported!
collected 1 item

tests/test_sum.py .

So, the import resolves because of an accident - the code under test is accidentally matching the test filename pattern. If you rename the module to something that's not matching the default discovery patterns, such as script_to_taste.py, then the test run will fail. Similarly, when you override the default patterns by adding python_files = test_* in pytest.ini you're disabling the *_test.py pattern and preventing script_to_test.py from being imported during the test discovery process.

Answer 2

following your structure:

from .. import script_to_test

def test_sum():
    d=script_to_test.sum(6,5)
    assert d==11

As you are importing from a folder, when you put:

import script_to_test

this will work for every file you have inside test folder...but your script_to_test is not in the same folder...is one level before...so two dots (..) will bring you to the correct level related to this file.