When should I use pytest --import-mode importlib

Question

This is my project structure:

git_dir/
    root/
        __init__.py
        tests/
            __init__.piy
            test1.py
        foo_to_test/
            foo.py

I'm using pytest to test foo.py, and test1.py is as follows:

from foo_to_test.foo import func_to_test

def test1():
    assert something about func_to_test

From the terminal, i want to run

pytest tests

Now for the problem. When using --import-mode append or --import-mode prepend it adds 'git_dir' to PYTHONPATH. The problem is that 'git_dir' is not the project root.

I can add sys.path.append('git_dir/root') but the name 'git_dir' is different for other programmers working on other computers.
I've seen in the pytest documentation that using --import-mode importlib might solve my problem, but it doesn't seem to have any effect on my PYTHONPATH and i can't understand what it is doing.

So my questions are:

What --import-mode importlib is doing?
How can i automatically add my root to the path when testing so it will be the same for every programmer pulling this project?

Does this help ? https://note.nkmk.me/en/python-script-file-path/ — Thingamabobs, Aug 23 '20 at 09:58
No, i need my project root directory to be in the path when i call the test from pytest. It seems that both of your solutions only get me the cwd — dor00012, Aug 23 '20 at 11:21

score 1 · Accepted Answer · answered Feb 09 '21 at 06:18

This looks hard because it's not how it's supposed to work. It may be time to learn about packaging.

The Python module system is designed to look for modules on the sys.path directories, as you describe. This variable is to be populated by the user, somehow, and should preferably not be meddled with.

When a developer wants to use your package, they must make sure it is installed on the system they run the tests on.

If you want to make that easier for them, provide e.g. a project.toml file; then they can build your package, and install it with pip install /path/to/your/distribution-file.tgz.

More info about that here: https://setuptools.readthedocs.io/en/latest/userguide/quickstart.html#python-packaging-at-a-glance.

score -1 · Answer 2 · answered Oct 31 '22 at 12:36

Your problem is that you have __init__.py in the root directory. When pytest finds a test module, it goes to the parent folders until it finds the one that does not have __init__.py. That's going to be you a directory that's added to sys.path, see Choosing a test layout / import rules. So your root directory should NOT be a Python module.

Now about importlib and why you probably don't need it

--import-mode=importlib circumvents the standard Python way of using modules and sys.path. The exact reasons are unclear from the docs:

With --import-mode=importlib things are less convoluted because pytest doesn’t need to change sys.path or sys.modules, making things much less surprising.

And they also mention that this allows test modules to have non-unique names. But why would we want this?! It seems like with proper file structure everything works fine even if we mess with sys.path.

Anyway, overall the usage of this option doesn't sound convincing, especially given this in the official docs:

Initially we intended to make importlib the default in future releases, however it is clear now that it has its own set of drawbacks so the default will remain prepend for the foreseeable future.

When should I use pytest --import-mode importlib

2 Answers2

Now about importlib and why you probably don't need it