How to run all test script file inside a directory in a specific order using pytest

Question

In pycharm I have a test directory named Tests, inside it I have 3 test script file - test_1.py, test_2.py and test_3.py.

I want to execute above files in a specific order using pytest.

Order should be:

test_2.py, test_1.py, test_3.py

I have seen we can run test method inside a test script file in a specific order using markers but I don't know how we can run test file in a specific order.

NOTE- I have given example of only 3 test file. In my project I have around 25 test file so I need a generic solution that can be applied upon all the test files.

Usually, the order of tests execution should not matter. Can you explain why do you need to execute them in specific order? There can be another solution to your problem. — Virtuoz, Dec 27 '21 at 07:02
Does this answer your question? [Test case execution order in pytest](https://stackoverflow.com/questions/17571438/test-case-execution-order-in-pytest) — zr0gravity7, Dec 27 '21 at 07:04
@Virtuoz I have some dependent test method inside test_2.py file that needs to be executed first. And that goes for all the test file they are interdependent. That is why I wanted to execute in a specific order. — Gaurav Chauhan, Dec 27 '21 at 07:10
@zr0gravity7 this solution is good for 2 or 3 files. But I have multiple files (I gave example of just 3 files. In my project I have around 25 files). So I wanted a generic solution which could be apply for all test files — Gaurav Chauhan, Dec 27 '21 at 07:13
@GauravChauhan I would suggest looking into pytest fixtures: https://docs.pytest.org/en/6.2.x/fixture.html. It seems that they can solve your problem in a much cleaner way. — Virtuoz, Dec 27 '21 at 07:19
Just add the tests explicitely to the command line, e.g. in your example `pytest test_2.py test_1.py test_3.py`, and they will be executed in that order. — MrBean Bremen, Dec 27 '21 at 09:54

score 0 · Answer 1 · answered Dec 28 '21 at 19:58

If you cannot remove the test dependencies (which is always the best solution), the easiest way to do this is probably to just call the tests on the command line in the order you need, e.g. in your example something like:

python -m pytest test2.py test1.py test3.py

If that is not an option, you can use test markers with pytest-order, as mentioned. In case you want to order only the test modules, but not the tests inside the modules, you can add a mark to the first test in each module, and use the --order-group-scope="module" argument in the pytest call. Given you have the test modules as given in the question, you would have to do something like:

test_2.py

@pytest.mark.order(0)
def test_something():
   ...

def test_something_else():
   ...

... # more tests

test_1.py

class TestSomething:
    @pytest.mark.order(1)
    def test_1(self):
        ...

... # more tests

test_3.py

@pytest.mark.order(2)
def test_something():
   ...

... # more tests

and then run pytest using for example:

python -m pytest -vv --order-group-scope="module"

This will first order the tests inside the modules (e.g. in this case do nothing), and than sort the modules themselves based on the markers.

Disclaimer:
I'm the maintainer of pytest-order.

How to run all test script file inside a directory in a specific order using pytest

1 Answers1