7

When using the hypothesis library and performing unit testing, how can I see what instances the library is trying on my code?

Example

from hypothesis import given
import hypothesis.strategies as st

@given(st.integers())
def silly_example(some_number):
    assert some_number > 0

The question is: how do I print / see the some_number variable, generated by the library?

Newskooler
  • 3,973
  • 7
  • 46
  • 84

4 Answers4

7

See here - either the note function and --hypothesis-verbosity=verbose, or the event function and --hypothesis-show-statistics should do the trick.

Zac Hatfield-Dodds
  • 2,455
  • 6
  • 19
  • 1
    Where do I write the `--hypothesis-verbosity=verbose`? I tried and still with no luck. – Newskooler Nov 05 '18 at 23:14
  • 1
    Like `pytest --hypothesis-verbosity=verbose my_tests.py` - if you're not using Pytest, you can use [`@settings` as a decorator, or the profiles mechanism](https://hypothesis.readthedocs.io/en/latest/settings.html). – Zac Hatfield-Dodds Nov 06 '18 at 08:50
  • I get an error when I do this though: `usage: pytest [options] [file_or_dir] [file_or_dir] [...]` and `pytest: error: unrecognized arguments: --hypothesis-verbosity=verbose` – Newskooler Nov 06 '18 at 14:37
  • 1
    Ah, you probably just need to update Hypothesis - the option was added in version 3.79.0, on 2018-10-23. – Zac Hatfield-Dodds Nov 13 '18 at 04:15
2

You could put a print statement or logging statement before the assert:

import logging
from hypothesis import given
import hypothesis.strategies as st

log_filename = 'debug.log'
logging.basicConfig(filename=log_filename, level=logging.DEBUG)
logger = logging.getLogger(__name__)

@given(st.integers())
def silly_example(some_number):
    logger.debug('silly_example(%s) called', some_number)
    assert some_number > 0

By using logging instead of print statements, you can turn off all logging simply by changing the logging level. If you change logging.DEBUG to logging.INFO:

logging.basicConfig(filename=log_filename, level=logging.INFO)

then logger.debug will no longer emit records.

unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • When formatting the logger function like so, I get a sonarcube error: `Use % formatting in logging functions and pass the % parameters as arguments `. Maybe it's best to use another formatting? – Newskooler Oct 31 '18 at 18:34
  • 1
    In that case, try: `logger.debug('silly_example(%s) called', some_number)`. – unutbu Oct 31 '18 at 18:41
  • 2
    The problem with logging or printing from a Hypothesis test is that the output does not distinguish between test cases, so it can be hard to tell which lines came from a particular failing case. That's why hypothesis provides the `note` and `event` functions to emit information that it can filter or summarise for you. – Zac Hatfield-Dodds Nov 02 '18 at 09:24
0

--hypothesis-verbosity=debug helps to see the output.

Aparna Bose
  • 31
  • 6
0

--hypothesis-verbosity=debug and --hypothesis-verbosity=verbose both works for me - but I have to get pytest not to capture the output as well, with a -s in front. Looks like this:

python -m pytest -s --hypothesis-verbosity=debug tests
kschnack
  • 362
  • 4
  • 12