0

I am a heavy, heavy, user of diffing-approaches to testing, so I tend to feed complex multi-line text strings to unittest.assertEqual. Typical of this type of use would be feeding a big chunk of html, which I will typically pre-process with beautifulsoup.prettify

What I have found is that, when the strings are sufficiently big and sufficiently different, assertEqual will spend either spend a huge amount of time, or just plain not return. If I ctrl-c the process, I will often find that it was interrupted in difflib.py. Sometimes, a pkill -laf 'ython.+test' in another terminal is needed.

Is that a simple Python approach that would me to conceptually execute:

unittest.assertEqual(a,b, timeout=5)

where that means after 5 seconds, terminate and tell me they're not equal and you couldn't get more details.

Of course, I assume that if I pass in the assertEqual.msg parameter, the function will not call difflib.py. Interestingly, while the diff command line utility typically returns a result after less than a second, a less clever differ, like Kaleidoscope on macOS, will often hang on those diffs as well.

Ideally, https://pypi.org/project/func-timeout/ would do exactly what I want. But it's LGPL, so not an option for my MIT application.

I already have some similar, thread-based, code to func_timeout, used to terminate an SQL connection attempt after a suitable delay. Guess I could use that. I could also launch a subprocess.popen(diff,..., timeout=5) to call the OS's native diff, since my strings are typically saved to file as well.

Seems to me like it's a problem that people pushing assertEqual a bit would frequently encounter, so wondering what approaches people have taken.

JL Peyret
  • 10,917
  • 2
  • 54
  • 73
  • Have you tried to use a timeout decorator ? Like this [one](https://pypi.org/project/timeout-decorator/) - There is also an [SO question](https://stackoverflow.com/questions/35490555/python-timeout-decorator) about timeout decorator, if it's what you're looking for – Arnaud Claudel Aug 14 '19 at 09:29
  • txs. will check it out. MIT licensed :-) – JL Peyret Aug 14 '19 at 09:39

1 Answers1

0

You could work around the whole issue by using unittest.assertTrue(a == b, "strings not equal") which avoids using diff.

blues
  • 4,547
  • 3
  • 23
  • 39
  • True, and that’s what I do when I notice it’s getting stuck. Have a command line option for it in fact. But... still not the greatest unitest behavior, is it? The diff’s quite helpful on small deltas, it’s likely not whenever it can’t resolve. – JL Peyret Aug 14 '19 at 09:37
  • It's not ideal. It wouldn't be hard though to extend unittest to provide an assert which only does this for long strings. something like `if len(a)+len(b) > 1000: assertTrue(...) else: assertEquals(...)` @JLPeyret – blues Aug 15 '19 at 10:53
  • Yes. But most of the time even big strings resolve quickly, on all tests. Problems only happen when say html structure has changed by an extra div, resulting in an indent across say 500 lines that now mismatch. Timeouts keep all functionality, avoid messing with tests. That’s this question, not otherwise valid workarounds. – JL Peyret Aug 15 '19 at 16:02