3

my question is - is it possible to use return value from fixture as a value in parametrize? The problem is - I'd like to dynamically get possible values (for example, available systems on a virtual server) for parametrize. I can access these when a virtual server is created by one of the fixtures. Tests look like this (pseudo-code-ish):

[conftest.py]

@pytest_fixture(scope='session')
def test_server(request):
    test_server = Server([default_params])
    test_server.add()
    def fin():
        test_server.delete()
    request_addfinalizer(fin)
    return test_server()

[tests.py]

def test_basic_server(test_server):
    systems = test.server.get_available_systems()
    for system in systems:
        test_server.install(system)
        test_server.run_checks()
        test_server.uninstall(system)

def test_other(test_server):
    [other tests]

etc

This way, one server is added for each session, then all tests run on it, and after session ends, server is removed. But is there a way to get the available systems in @pytest.mark.parametrize without explicitly listing them (statically as a list in parametrize), using the method from server that is added when the session begins? That way each system would run in a separate test.

I tried using test_server in another fixture and then returning the list (the same way test_server is returned by test_server fixture, but I cannot use that as a value in parametrize - since decorator is evaluated before the test_server fixture is called in any test, and getting the list depends on test_server fixture.

This would be ideal:

[tests.py]

@pytest.mark.parametrize('system',[systems_list <- dynamically generated
                                             when the server is created])

def test_basic_server(test_server,system):
    test_server.install(system)
    test_server.run_checks()
    test_server.uninstall(system)

This is just a very basic example, in my tests I need to parametrize based on multiple scenarios and values and I end up with giant arrays when I do it statically.

But the principle remains the same - basically: can I call the fixture before the first test using this fixture runs, or how can pytest.mark.parametrize() access fixture values?

Taku
  • 562
  • 1
  • 6
  • 15

2 Answers2

2

I think you may be unable to achieve what you want directly. Because @pytest.mark.parametrize is being called during collecting, and fixtures will be called after collection completed.

But I have an alternative way to achieve similar result, mainly by extending pytest plugin pytest_generate_tests and using method metafunc.parametrize. https://pytest.org/latest/parametrize.html#basic-pytest-generate-tests-example

Here is my solution. In conftest.py

class System(object):
    def __init__(self, name):
        self.name = name

    def __repr__(self):
        return "<System '{}'>".format(self.name)

def get_available_systems():
    return [System('A'), System('B'), System('C')]


def pytest_generate_tests(metafunc):
    if 'system' in metafunc.fixturenames:
        available_systems = get_available_systems()

        metafunc.parametrize('system', available_systems)

In test file:

def test_basic_server(system):
    print(system)

This is the output, you will have access to each system in test.

collected 3 items

test_01.py::test_basic_server[system0] <System 'A'>
PASSED
test_01.py::test_basic_server[system1] <System 'B'>
PASSED
test_01.py::test_basic_server[system2] <System 'C'>
PASSED

The bad thing is, get_available_systems will be called every time whenever using fixture system, which is not what you want. But I think it's not hard to add some extra logic to make the query logic only be executed once.

For example:

def pytest_generate_tests(metafunc):
    if 'system' in metafunc.fixturenames:
        if hasattr(metafunc.config, 'available_systems'):
            available_systems = metafunc.config.available_systems
        else:
            available_systems = get_available_systems()
            metafunc.config.available_systems = available_systems
        metafunc.parametrize('system', available_systems)
Li Feng
  • 931
  • 6
  • 14
  • Yes, I've been looking into metafunc, and this is a valid approach, but still, that would demand either adding another server before the test begins just to collect the systems list. IRL I have to consider multiple methods (more than 20), which return dynamic lists based on what's available. On average, that means more than 300 cases, which now run as a single test - so you can see why it would be nice if parametrize just could swallow the list and generate separate cases. Nevertheless, I'll look deeper into it, maybe with metafunc I'll get somewhere. Thank you :) – Taku Jun 23 '16 at 07:58
  • Actually, if I managed to add (physically) a server during evaluation of conftest.py, then accessed its methods to get the lists I need, stored them and then use in pytest_generate_tests(), on the surface that would be exactly what I needed :) Not the cleanest solution, but that would get the job done, and I wouldn't have to maintain hundreds of variables that could change at any given moment. – Taku Jun 23 '16 at 10:15
  • Yes I think it's a good idea to run test against different server as separate tests (by using `parametrize`) , which makes test result clearer and finding failure cause easier. Also hope to know any neater solution – Li Feng Jun 23 '16 at 18:39
0

I was able to work a similar problem where I have to generate tests data that is to be used for parameterization on the fly:

class TestFilters(object):
 cls_testdata1 = []

def setup_class(cls):
 r = []
 for i in range(5):
   r.append((x, y, z))
 TestFilters.cls_testdata1 = r

@pytest.mark.parametrize("filter_id", list(range(5)))
def test_func(self, filter_id):
    params = TestFilters.cls_testdata1[filter_id]

...

This would support adding the parameters dynamically, only that you need to predetermine the number of tests.

MyDevMallik
  • 35
  • 6