5

I am using pytest framework and want to skip testcase based on some condition. Below code is not skipping the test case. 

import pytest
class TEST:
    @pytest.fixture(scope="function", autouse=True)
    def func_fixture(self):
        return "fail"

    @pytest.mark.skipif("self.func_fixture=='fail'")
    def test_setting_value(self):
        print("Hello I am in testcase")

When running, it is saying 0 testcases executed.

This func_fixture is very crucial for the testsuite. It performs many pre-requisites before starting the test.

If I remove class, and add rest of the functions with same syntax (after removing self), it works. Not sure why it is failing in class

hoefling
  • 59,418
  • 12
  • 147
  • 194
Sumit
  • 1,953
  • 6
  • 32
  • 58

4 Answers4

2

First, according to Conventions for Python test discovery, the class name should starts with Test:

From those files, collect test items:

  • test_ prefixed test functions or methods outside of class
  • test_ prefixed test functions or methods inside Test prefixed test classes (without an __init__ method)
class Test...:

Second, func_fixture in @pytest.mark.skipif("func_fixture=='fail'") is a function, not a return value of the function call. (I don't know how to use fixture value inpytest.mark.skipif(..); See this answer to see how to use fixture in expression for skipif).

import pytest


@pytest.fixture
def func_fixture():
    return 'fail'


skip_by_fixture = pytest.mark.skipif("func_fixture() == 'fail'")

class TestSetting:
    @skip_by_fixture
    def test_setting_value(self):
        print("Hello I am in testcase")
falsetru
  • 357,413
  • 63
  • 732
  • 636
  • You have moved fixture `func_fixture` out of class. What if I want to put this fixture inside class – Sumit Aug 09 '18 at 08:44
  • Your code is working but I want the fixture func_fixture inside class. – Sumit Aug 09 '18 at 08:49
  • @Nitesh, Sorry, I don't know how inject the fixture value into the expression for skipif / using instance method as a fixture. – falsetru Aug 09 '18 at 09:52
  • @Nitesh, How about this: https://pastebin.com/Hw2C1kKa (`func_fixture` is not a fixture, just a static method) – falsetru Aug 09 '18 at 10:08
  • now func_fixture is not a fixture. I am having actions that I am performing in a fixture. The idea is that if preconditions are not satisfied I don't want to execute testcase – Sumit Aug 09 '18 at 10:44
  • @Nitesh, Even though it's not a fixture: It does what you want: Skip the test case if condition is not satisfied ;) Change the `func_fixture` to return `'success'` and try `pytest`. – falsetru Aug 09 '18 at 14:03
  • @falsetru : How can I do it, if have to update 'fail' to 'pass' on certain criteria. And later skip rest of the test cases if it's 'pass'. – StackGuru Jul 12 '20 at 15:09
  • https://stackoverflow.com/questions/62857682/how-to-skip-tests-based-on-condition-in-pytest?noredirect=1#comment111156247_62857682 – StackGuru Jul 12 '20 at 15:09
2

TL;DR

you can skip on condition inside the test itself; skip to Suggestion at the end of the answer. With your example code, skipping tests via skipIf markers is not possible.

First of all, let me clear the following: if you define an autouse fixture foo that returns a value, it does not mean the test class will automatically become an attribute foo.

class TestCls:

    @pytest.fixture(autouse=True)
    def fix(self):
        return 'foo'

    def test_fix(self):
        assert self.fix == 'foo'

Running this test will fail:

    def test_fix(self):
>       assert self.fix == 'foo'
E       AssertionError: assert <bound method TestCls.fix of <test_bacon.TestCls object at 0x105670cf8>> == 'foo'
E        +  where <bound method TestCls.fix of <test_bacon.TestCls object at 0x105670cf8>> = <test_bacon.TestCls object at 0x105670cf8>.fix

That's because that's not how fixtures work. If a fixture is returning something, you can access its return value if you add a parameter named same as the fixture, in the test function. This is a problem with classes because you can't pass fixtures as parameters to test class methods; overall, the support of test classes in pytest is limited. Not sure why you need classes at all; you can perfectly share the state between multiple test functions using fixtures alone. The whole intention of pytest is to make test classes obsolete, where every test method can be reworked to a test function.

If you need to share a value between tests in a class, you need to assign it to some attribute of the test class instance in the fixture. The example below will pass:

class TestCls:

    @pytest.fixture(autouse=True)
    def fix(self):
        self._fix = 'foo'

    def test_fix(self):
        assert self._fix == 'foo'

Thus, fixtures that actually return something are useless in test classes, because there's no way to pass fixtures as arguments in test class methods.

Not sure why it is failing in class

That's because the markers are evaluated on tests collection to filter out tests that should be executed, and fixtures are executed only after the tests are collected and ready to run. This means what you want is not possible. You will not be able to pass any fixture result to skipif marker as no fixture is evaluated yet. Check the execution order:

@pytest.fixture(autouse=True)
def myfixture():
    print('myfixture called')


class TestCls:

    @pytest.fixture(autouse=True)
    def myclassfixture(self):
        print('TestCls.myclassfixture called')

    @pytest.mark.skipif('print(os.linesep, "TestCls.test_spam skipif called")', reason='something')
    def test_spam(self):
        print('TestCls.test_spam called')

Output:

TestCls.test_spam skipif called
myfixture called
TestCls.myclassfixture called
TestCls.test_spam called

Also note that the test class instance is not available in skipif, only what's defined on module level (classes, functions and global variables):

class Tests:

    @pytest.mark.skipif('print(os.linesep, "globals:", globals().keys(), os.linesep, "locals:", locals().keys())', reason='something')
    def test_spam(self):
        pass

Output:

test_spam.py::TestCls::test_spam
 globals: dict_keys(['os', 'sys', 'platform', 'config', '__name__', '__doc__', '__package__', '__loader__', '__spec__', '__file__', '__cached__', '__builtins__', '@py_builtins', '@pytest_ar', 'pytest', 'myfixture', 'TestCls'])
 locals: dict_keys(['os', 'sys', 'platform', 'config', '__name__', '__doc__', '__package__', '__loader__', '__spec__', '__file__', '__cached__', '__builtins__', '@py_builtins', '@pytest_ar', 'pytest', 'myfixture', 'TestCls'])

Suggestion

Besides the skip/skipIf markers, you can skip the test explicitly inside the test itself:

class TestCls:

    @pytest.fixture(autouse=True)
    def myfixture(self):
        self.status = 'fail'

    def test_eggs(self):
        if self.status == 'fail':
            pytest.skip('self.status is set to "fail"')
        assert False

If run, the test is skipped:

test_eggs.py::TestCls::test_eggs SKIPPED
hoefling
  • 59,418
  • 12
  • 147
  • 194
  • Thanks for this solution. If I want to update the value of status in any of the testcases to 'pass' , how can I do that ? – StackGuru Jul 12 '20 at 15:00
  • In one of testcases, i'm updating the value as if some_condition then self.status='pass' , But when i check in next test case as if self.status == 'pass', it still has 'fail' – StackGuru Jul 12 '20 at 15:03
  • https://stackoverflow.com/questions/62857682/how-to-skip-tests-based-on-condition-in-pytest?noredirect=1#comment111156247_62857682 – StackGuru Jul 12 '20 at 15:09
1

Pytest provides a feature to manage dependencies of tests. You can create a separate test and set dependency on successive tests. You need to install pytest-dependency.

pip install pytest-dependency

Example:

class TestExample(object):

    @pytest.mark.dependency()
    def test_func(self):
        assert False

    @pytest.mark.dependency(depends=["TestExample::test_func"])
    def test_setting_value(self):
        print("Hello I am in testcase")
Parth Naik
  • 186
  • 1
  • 6
  • Your answer is not very clear. Can you explain it more – Sumit Aug 09 '18 at 09:06
  • You can assert your condition which you've used in your fixture in a seperate test, say test_func(in above example). If the `test_func` passes, only then `test_setting_value` will be executed, else that test will be skipped, as we have set dependency of test_func on this test. – Parth Naik Aug 09 '18 at 09:10
  • `test_func` is actually a fixture which perform certain steps before a test case start. can @pytest.mark.dependency be merged with fixtures too ? – Sumit Aug 09 '18 at 09:13
  • It only allows to mark some tests as dependent from other tests. These tests will then be skipped if any of the dependencies did fail or has been skipped. But in your case, you can replace your fixture with a test which will be executed first and set dependency on other tests. [https://pytest-dependency.readthedocs.io/en/latest/usage.html#basic-usage] – Parth Naik Aug 09 '18 at 09:35
1

First, class name with just TEST will not work.

Second, fixture sometimes is a pain ..., I would just use a plain method to check and return condition to skip(fail) or not.

import pytest
import unittest


class TestSkipWithoutFixture(unittest.TestCase):

    def fixture_sometimes_suck():
        return "fail"

    @pytest.mark.skipif(fixture_sometimes_suck() == 'fail', reason='just skip')
    def test_setting_value(self):
        print("Hello I am in testcase")

    def test_to_pass(self):
        pass

test results pytest -v tests/test_skip.py

platform linux2 -- Python 2.7.14, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python2

collected 2 items                                                                                                                                 

tests/test_skip.py::TestSkipWithoutFixture::test_setting_value SKIPPED
tests/test_skip.py::TestSkipWithoutFixture::test_to_pass PASSED
Gang
  • 2,658
  • 3
  • 17
  • 38
  • If there's a condition check inside 'test_setting_value' , if a == 1, then rest of the testcases should be skipped. Is that possible ? https://stackoverflow.com/questions/62847026/how-can-we-halt-stop-fail-test-cases-if-a-condition-fails-in-another-test-case?noredirect=1#comment111137349_62847026 – StackGuru Jul 11 '20 at 10:48
  • @StackGuru, fasetru's answer is closed related to your question, there is no order which test run first in a TestClass, so the condition has to be outside of a test class, a fixture if preferred if the same condition for all tests inside a class, but still, the skipif fixture has to be on top of each test func – Gang Jul 12 '20 at 13:40
  • how can i update "fail" to "pass" on certain criteria in one of the testcases. Later add marker skipif accordingly. – StackGuru Jul 12 '20 at 15:05
  • https://stackoverflow.com/questions/62857682/how-to-skip-tests-based-on-condition-in-pytest?noredirect=1#comment111156247_62857682 – StackGuru Jul 12 '20 at 15:09