PyCharm show full diff when unittest fails for multiline string?

Question

I am writing some Python unit tests using the "unittest" framework and run them in PyCharm. Some of the tests compare a long generated string to a reference value read from a file. If this comparison fails, I would like to see the diff of the two compared strings using PyCharms diff viewer.

So the the code is like this:

    actual = open("actual.csv").read()
    expected = pkg_resources.resource_string('my_package', 'expected.csv').decode('utf8')
    self.assertMultiLineEqual(actual, expected)

And PyCharm nicely identifies the test as a failure and provides a link in the results window to click which opens the diff viewer. However, due to how unittest shortens the results, I get results such as this in the diff viewer:

Left side:

'time[57 chars]ercent 0;1;1;1;1;1;1;1 0;2;1;3;4;2;3;1 0;3;[110 chars]32 '

Right side:

'time[57 chars]ercen 0;1;1;1;1;1;1;1 0;2;1;3;4;2;3;1 0;3;2[109 chars]32 '

Now, I would like to get rid of all the [X chars] parts and just see the whole file(s) and the actual diff fully visualized by PyCharm.

I tried to look into unittest code but could not find a configuration option to print full results. There are some variables such as maxDiff and _diffThreshold but they have no impact on this print.

Also, I tried to run this in py.test but there the support in PyCharm was even less (no links even to failed test).

Is there some trick using the difflib with unittest or maybe some other tricks with another Python test framework to do this?

This isn't related to PyCharm, nor to py.test. The same things happens when running Python3.4 unittests from the command line, using plain old "python -m unittest" — Jonathan Hartley, Jun 04 '15 at 12:34
Neither is it related to assertMultiLineEqual - I'm seeing the same behaviour for plain old assertEqual. — Jonathan Hartley, Jun 04 '15 at 12:34
The abbreviated (missing) parts of each string are not guaranteed to be equal in both strings (if they were, this behaviour might be mostly benign). But the example shown above in the question shows the first string to have [110 chars] omitted where the 2nd string has [109 chars] omitted. So the omitted strings can't possibly be equal. Why abbreviate if the missing portion contains part of the diffs!? — Jonathan Hartley, Jun 04 '15 at 12:35

score 13 · Answer 1 · edited May 23 '17 at 12:30

13

The TestCase.maxDiff=None answers given in many places only make sure that the diff shown in the unittest output is of full length. In order to also get the full diff in the <Click to see difference> link you have to set MAX_LENGTH.

import unittest

# Show full diff in unittest
unittest.util._MAX_LENGTH=2000

Source: https://stackoverflow.com/a/23617918/1878199

edited May 23 '17 at 12:30

Community

1
1

answered Dec 06 '15 at 12:03

ootwch

930
11
32

2

Also works if you put it into the `def setUp(self):` if you want no code outside your test class. – luckydonald Jul 01 '16 at 14:42
2

This does not work for me at all. In which version of PyCharm are you testing this? – Mitar Apr 19 '19 at 22:08

score 4 · Accepted Answer · answered Jun 09 '15 at 10:32

Well, I managed to hack myself around this for my test purposes. Instead of using the assertEqual method from unittest, I wrote my own and use that inside the unittest test cases. On failure, it gives me the full texts and the PyCharm diff viewer also shows the full diff correctly.

My assert statement is in a module of its own (t_assert.py), and looks like this

def equal(expected, actual):
    msg = "'"+actual+"' != '"+expected+"'"
    assert expected == actual, msg

In my test I then call it like this

    def test_example(self):
        actual = open("actual.csv").read()
        expected = pkg_resources.resource_string('my_package', 'expected.csv').decode('utf8')
        t_assert.equal(expected, actual)
        #self.assertEqual(expected, actual)

Seems to work so far..

patricksurry · Answer 3 · 2021-11-08T14:34:12.680

A related problem here is that unittest.TestCase.assertMultiLineEqual is implemented with difflib.ndiff(). This generates really big diffs that contain all shared content along with the differences. If you monkey patch to use difflib.unified_diff() instead, you get much smaller diffs that are less often truncated. This often avoids the need to set maxDiff.

import unittest
from unittest.case import _common_shorten_repr
import difflib


def assertMultiLineEqual(self, first, second, msg=None):
    """Assert that two multi-line strings are equal."""
    self.assertIsInstance(first, str, 'First argument is not a string')
    self.assertIsInstance(second, str, 'Second argument is not a string')

    if first != second:
        firstlines = first.splitlines(keepends=True)
        secondlines = second.splitlines(keepends=True)
        if len(firstlines) == 1 and first.strip('\r\n') == first:
            firstlines = [first + '\n']
            secondlines = [second + '\n']
        standardMsg = '%s != %s' % _common_shorten_repr(first, second)
        diff = '\n' + ''.join(difflib.unified_diff(firstlines, secondlines))
        standardMsg = self._truncateMessage(standardMsg, diff)
        self.fail(self._formatMessage(msg, standardMsg))

unittest.TestCase.assertMultiLineEqual = assertMultiLineEqual

This worked perfectly for me! – Grayson Pike Feb 09 '22 at 21:42 — Grayson Pike, Feb 09 '22 at 21:42

PyCharm show full diff when unittest fails for multiline string?

3 Answers3

Linked