Explicitly defined test case execution order in pytest

Question

I have some scripts with the tests for them and I need to run these tests in execution order explicitly defined by me.

It looks like:

# one.py
import some lib

class Foo():
    def makesmth(self)
        script

then I have made test files:

# test_one.py
import pytest
import some lib

class TestFoo():
    def test_makesmth(self):
        try/except/else assert etc.

So it looks simple and right. When I run file test_one.py everything is ok. Package of my scripts-testing looks like:

package/
|-- __init__.py
|-- scripts
|   |-- one.py
|   |-- two.py
|-- tests
|   |-- test_one.py
|   |-- test_two.py

When I try to collect test with

pytest --collect-only

it is giving non-alphabetical and just randomly order of tests.

Where I can write information about order of tests? Non-alphabetical, just like I want to start test like b, a, c, e, d - and not random not alphabetical

Tried to made file tests.py:

import pytest

from tests.test_one import TestFoo
from tests.test_two import TestBoo etc.

And when I'm trying to run this, errors are shown, because these imports were done in the way I don't understand (tried to make aTestFoo bTestBoo and also rename test files in that method definition way but still it's doesn't work).

score 4 · Answer 1 · answered Mar 23 '21 at 15:31

4

Pytest-ordering seems abandoned at the moment, you can also check out pytest-order (a fork of the original project).

import pytest

@pytest.mark.order(2)
def test_foo():
    assert True

@pytest.mark.order(1)
def test_bar():
    assert True

answered Mar 23 '21 at 15:31

Roelant

4,508
1
32
62

score 2 · Answer 2 · answered Dec 24 '18 at 14:04

You can use pytest-ordering

See https://pytest-ordering.readthedocs.io/en/develop/

import pytest
@pytest.mark.run(order=1)
def test_first():
    pass
@pytest.mark.run(order=2)
def test_second():
    pass

test_sample.py::test_first PASSED
test_sample.py::test_second PASSED

score 0 · Answer 3 · answered Jan 18 '22 at 16:41

If you don't want to use external libraries, you can use the pytest_collection_modifyitems hook to modify the order of collected tests in place.

With custom implementations, one can use the test name, test class, test module or test sub-directory to enforce a test case execution order. In your particular case, it seems like enforcing an execution order by test class would make the most sense. A full example on how to do it is already available in this answer.

Explicitly defined test case execution order in pytest

3 Answers3