Let's say I have a bunch of tests:
def test_func_one():
...
def test_func_two():
...
def test_func_three():
...
Is there a decorator or something similar that I could add to the functions to prevent pytest from running just that test? The result might look something like...
@pytest.disable()
def test_func_one():
...
def test_func_two():
...
def test_func_three():
...
Pytest has the skip and skipif decorators, similar to the Python unittest module (which uses skip and skipIf), which can be found in the documentation here.
Examples from the link can be found here:
@pytest.mark.skip(reason="no way of currently testing this")
def test_the_unknown():
...
import sys
@pytest.mark.skipif(sys.version_info < (3,3),
reason="requires python3.3")
def test_function():
...
The first example always skips the test, the second example allows you to conditionally skip tests (great when tests depend on the platform, executable version, or optional libraries.
For example, if I want to check if someone has the library pandas installed for a test.
import sys
try:
import pandas as pd
except ImportError:
pass
@pytest.mark.skipif('pandas' not in sys.modules,
reason="requires the Pandas library")
def test_pandas_function():
...
The skip decorator would do the job:
@pytest.mark.skip(reason="no way of currently testing this")
def test_func_one():
# ...
(reason argument is optional, but it is always a good idea to specify why a test is skipped).
There is also skipif() that allows to disable a test if some specific condition is met.
These decorators can be applied to methods, functions or classes.
To skip all tests in a module, define a global pytestmark variable:
# test_module.py
pytestmark = pytest.mark.skipif(...)
You can mark a test with the skip and skipif decorators when you want to skip a test in pytest.
@pytest.mark.skip(reason="no way of currently testing this")
def test_func_one():
...
The simplest way to skip a test is to mark it with the skip decorator which may be passed an optional reason.
It is also possible to skip imperatively during test execution or setup by calling the pytest.skip(reason) function. This is useful when it is not possible to evaluate the skip condition during import time.
def test_func_one():
if not valid_config():
pytest.skip("unsupported configuration")
@pytest.mark.skipif(sys.version_info < (3, 6), reason="requires python3.6 or higher")
def test_func_one():
...
If you want to skip based on a conditional then you can use skipif instead. In the previous example, the test function is skipped when run on an interpreter earlier than Python3.6.
Finally, if you want to skip a test because you are sure it is failing, you might also consider using the xfail marker to indicate that you expect a test to fail.
If you want to skip the test but not hard code a marker, better use keyword expression to escape it.
pytest test/test_script.py -k 'not test_func_one'
Note: Here 'keyword expression' is basically, expressing something using keywords provided by pytest (or python) and getting something done. In above example, 'not' is a keyword.
For more info, refer this link.
More examples of keyword expression can be found in this answer.
I'm not sure if it's deprecated, but you can also use the pytest.skip function inside of a test:
def test_valid_counting_number():
number = random.randint(1,5)
if number == 5:
pytest.skip('Five is right out')
assert number <= 3
You may also want to run the test even if you suspect that test will fail. For such scenario https://docs.pytest.org/en/latest/skipping.html suggests to use decorator @pytest.mark.xfail
@pytest.mark.xfail
def test_function():
...
In this case, Pytest will still run your test and let you know if it passes or not, but won't complain and break the build.
You can divide your tests on set of test cases by custom pytest markers, and execute only those test cases what you want. Or the inverse, running all tests except the another set:
@pytest.mark.my_unit_test
def test_that_unit():
...
@pytest.mark.my_functional_test
def test_that_function():
...
And then, to run only one set of unit tests, as example:
pytest -m my_unit_test
Inverse, if you want to run all tests, except one set:
pytest -m "not my_unit_test"
More examples in official documentation
It looks more convenient if you have good logic separation of test cases.
If you like to disable an entire testclass, you can use the __test__ = False
We use it in large testsuites for spesific test that we only run on special occations, like long lasting regression tests or tests that always fails.
We then set an environment variable like:
export SCIM_GROUP_REGRESSION_TEST=1
if we want to run it
SCIM_GROUP_REGRESSION_TEST = False
if os.environ.get('SCIM_GROUP_REGRESSION_TEST'):
SCIM_GROUP_REGRESSION_TEST = True
class TestClass:
__test__ = SCIM_GROUP_REGRESSION_TEST
:
<your tests>
Then you explicitly need to set this environment-variable to run it, in all other occations it will be skipped
People have spoken of the skip decorator. Works like a charm!
A better way to set this up is with the skipif decorator which allows you to skip certain tests conditionally.
This is a construct I love to use especially for long tests.
import pytest
import os
# your skipif decorator to conditionally skip a test that takes too long
@pytest.mark.skipif("RUNALL" not in os.environ, reason="takes too long")
def test_a_very_long_process():
# Your test code...
What this does is that it will check that the environment variable RUNALL exists. If not, it will skip those tests.
This means that the usual pytest command will skip those tests. However, when the time to run all tests (like when going into production mode) comes, you can run RUNALL=true pytest.
Of course you can evaluate any other conditions with skipif :)
