How to organize steps from several test functions when using pytest?

Question

I have three test cases with some dependency of two of them on the third one. Namely, tests test_inner_1 and test_inner_2 are independent from each other but their execution makes no sense if test_outher fails. They both should be run if test test_outher passes and both should be skipped if test_outher fails.

The pytest manual https://pytest.org/latest/example/simple.html presents some simple example how to implement incremental testing with test steps. I am trying to apply this approach to my situation and to implement something like that:

content of conftest.py:

import pytest

def pytest_runtest_makereport(item, call):
    if "incremental" in item.keywords:
        if call.excinfo is not None:
            parent = item.parent
            parent._previousfailed = item


def pytest_runtest_setup(item):
    if "incremental" in item.keywords:
        previousfailed = getattr(item.parent, "_previousfailed", None)
        if previousfailed is not None:
            pytest.xfail("previous test failed (%s)" % previousfailed.name)

content of test_example.py:

import pytest

@pytest.mark.incremental
class TestUserHandling:
    def test_outher(self):
        assert 0

    class TestInner:
        def test_inner_1(self):
            assert 0

        def test_inner_2(self):
            pass

Unfortunately, I have got the output

==================== 2 failed, 1 passed in 0.03 seconds ====================

while expected to get the output

=================== 1 failed, 2 xfailed in 0.03 seconds ====================

How to correct the conftest.py to get the desired behaviour?

Possible duplicate of [If I have multiple tests in one class and a preceding test fails, how do I have it skip or exit the class instead of testing the remaining tests?](http://stackoverflow.com/questions/37422806/if-i-have-multiple-tests-in-one-class-and-a-preceding-test-fails-how-do-i-have) — M.T, May 29 '16 at 08:03
It is an another question. I need to group two tests to make them BOTH run/skipped in dependence of other test passed of failed. These two tests are independent on each other. — Andriy, May 30 '16 at 11:47

score 2 · Accepted Answer · answered Sep 13 '17 at 21:08

There is a plugin for pytest called pytest-dependency that does what you want to do in this case.

Your code can look like this:

import pytest
import pytest_dependency

@pytest.mark.dependency()
def test_outher():
   assert 0

@pytest.mark.dependency(depends=["test_outher"])
def test_inner_1():
    assert 0

@pytest.mark.dependency(depends=["test_outher"])
def test_inner_2():
    pass

Output is:

=================================== FAILURES ===================================
_________________________________ test_outher __________________________________

@pytest.mark.dependency()
def test_outher():
>      assert 0
E      assert 0

test_example.py:6: AssertionError
===================== 1 failed, 2 skipped in 0.02 seconds ======================

You can use classes of course, but for this example it is not necessary. If you need example with classes, let me know.

smarie · Answer 2 · 2018-10-11T17:02:02.823

You can also use pytest-steps to create a test suite containing your tests and declare the dependencies, it will behave the same way (skipping inner1 and inner2 if outher fails):

EDIT: new generator mode makes it even easier:

from pytest_steps import test_steps, optional_step

@test_steps('step_outher', 'step_inner_1', 'step_inner_2')
def test_suite():
    # Step outher
    assert 1
    yield

    # Step inner 1
    with optional_step('step_inner_1') as step_inner_1:
        assert 0
    yield step_inner_1

    # Step inner 2
    with optional_step('step_inner_2') as step_inner_2:
        assert 0
    yield step_inner_2

LEGACY answer:

from pytest_steps import test_steps, depends_on

def step_outher():
   assert 0

@depends_on(step_outher)
def step_inner_1():
    assert 0

@depends_on(step_outher)
def step_inner_2():
    pass

@test_steps(step_outher, step_inner_1, step_inner_2)
def test_suite(test_step):
    # Execute the step
    test_step()

You can also use it to share intermediate results among the test steps, see the documentation for details. (I'm the author by the way :) )

How to organize steps from several test functions when using pytest?

2 Answers2