2

I have a composite, expensive-to-build but cheap-to-test strategy. I must do:

@given(expensive_strategy())
def test_all(x):
    assert...
    assert...
    ...

It takes ~4 seconds to build the examples and negligible time to run the asserts.

Best practice dictates these tests be separated.

It is not clear to me how I could combine hypothesis strategies and eg. TestCase.setUp or pytest session-scoped fixtures. Both decorating the fixture and calling with x = expensive_strategy(); @given(x) does not help.

Eric Kaschalk
  • 369
  • 1
  • 4
  • 8
  • what about [`session`-scoped fixtures](https://docs.pytest.org/en/latest/fixture.html#sharing-a-fixture-across-tests-in-a-module-or-class-session)? – Azat Ibrakov May 11 '17 at 17:22

1 Answers1

2

One option would be to use the strategy solely for example generation, and ignore testcase shrinking, the examples database, etc. This will make a passing test suite faster, and a failing test suite more confusing. Something like the following should work:

class MyTest(TestCase):
    @classmethod  # or however this is done in your test runner
    def setUpClass(cls):
        strategy = expensive_strategy()
        cls.examples = [strategy.example() for _ in range(1000)]

    def test_all(self):
        for x in self.examples:
            assert invariant(x)

Another option would be to mangle hypothesis' internals so that each testcase receives the same bytestream, and do caching on whatever is slow in constructing the strategy, though that would surely violate "best practices".

Personally, I would just not follow "best practices" when they lead to absurd results like a slow test suite, and make all the assertions in a single test method, possibly via calls to helper methods like assert_a_particular_kind_of_assertion(x).

Lucas Wiman
  • 10,021
  • 2
  • 37
  • 41
  • 2
    The author of hypothesis pointed out that the first option is not a good strategy, since `.example()` is not representative of what `hypothesis` would do on an ordinary test run. – Lucas Wiman Aug 17 '17 at 01:25
  • When "best practices" lead to the absurd, resort to "best theories". – Chris Wesseling Mar 24 '23 at 10:53