1

I wrote a test to verify a given function behaves correctly for any given permutation of a list specified as input.

Using the hypothesis python package I've tried to construct this test case. However, the list of permutations generated contains many duplicate test cases.

For instance, using a list of 3 items (['a', 'b', 'c']) I would expect 6 entries in the list of permutations (abc, acb, bac, bca, cab, cba), but this is not the case.

Here's a MWE, assuming you have the hypothesis package:

from hypothesis import given
from hypothesis.strategies import permutations

@given(permutations(['a', 'b', 'c']))
def test(permutation):
    print(permutation)

test()

Alternatively, the code is live on repl.it.

The documentation unfortunately didn't enlighten me, but maybe I'm not looking in the right place.

Any idea on what I'm doing wrong?

Jir
  • 2,985
  • 8
  • 44
  • 66
  • The answer to [How do you generate dynamic (parameterized) unit tests in python?](https://stackoverflow.com/q/32899/222529) might be useful to others in a similar situation. – Jir Jul 03 '19 at 12:25

1 Answers1

1

You're not doing anything wrong, that's just how Hypothesis works. It doesn't (and can't) guarantee that it doesn't generate duplicate elements. It makes a best effort attempt to reduce their frequency, but it's not possible for it to do so in general, and the problem rarely comes up with more complex strategies, so it's not worth special casing the few cases where it is possible.

If the set of possible examples is small enough that you can just exhaustively enumerate it and it really matters to you to avoid duplicates, you might find something like pytest.mark.parametrize a better fit.

DRMacIver
  • 2,259
  • 1
  • 17
  • 17
  • After close inspection and a chat with some hypothesis user on freenode, I have to agree with you - that's how hypothesis works. I would only add that other solutions (on top of the ones you posted) are: a) to use itertools.permutations if the permutations space is small enough that can be enumerated exhaustively or b) other packages (you mentioned pytest, and there's also nosetests or even unittest.subTest). – Jir Jul 03 '19 at 12:19