Annoying diff format for long strings using pytest + Pycharm

Question

Hi have this very basic test:

def test_long_diff():
    long_str1 = "ABCDEFGHIJ " * 10
    long_str2 = "ABCDEFGHIJ " * 5 + "* " + "ABCDEFGHIJ " * 5
    assert long_str1 == long_str2

Using: Python 3.8.5, pytest-6.2.1, PyCharm 2020.2, MacOs

Running with pytest from a shell, the output is "useable" and the error message will point out the faulty character in the long string:

(venv) ~/dev/testdiff/> pytest longdiff.py 
========== test session starts ===========
platform darwin -- Python 3.8.5, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
[...]
>       assert long_str1 == long_str2
E       AssertionError: assert 'ABCDEFGHIJ A...J ABCDEFGHIJ ' == 'ABCDEFGHIJ A...J ABCDEFGHIJ '
E         Skipping 45 identical leading characters in diff, use -v to show
E         - BCDEFGHIJ * ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ 
E         ?          --
E         + BCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ

Using pytest-clarity and the -vv option, I get coloured diff (not rendered bellow) and different details:

E         left:  "ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ "
E         right: "ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ * ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ "
E         
E         left and right have different lengths:
E           len(left) == 110, len(right) == 112

But if I let Pycharm run the test (same Python version, same .venv, I just right-click the test and pick "Run 'pytest for ...'"), the output in the Run Console is almost unusable because "something along the way" turns the long strings into tuples of shorter strings before applying the diff:

FAILED                                       [100%]
longdiff.py:0 (test_long_diff)
('ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ '
 'ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ') != ('ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ * ABCDEFGHIJ '
 'ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ')

<Click to see difference>

Clicking <Click to see difference> basically shows the same garbled output in a larger window

What's causing this Pycharm output? And is there a way to prevent this behaviour ? Ideally I'd like to see the output from pytest-clarity in the Run console.

PyCharm bundles own `pytest` plugins that can't be turned off AFAIK. — hoefling, Dec 23 '20 at 16:07
I have no issue with pytest in PyCharm. So I would check your settings (on run configuration). Maybe in past you put some special setting — Giacomo Catenazzi, Dec 23 '20 at 16:54
Can you please be more specific @GiacomoCatenazzi ? I've already looked at all possible settings I could think of. Do you get a different output if you run my sample test in PyCharm ? What OS / Version are you using ? — Coyotwill, Dec 23 '20 at 23:07
Indeed @hoefling, I found [the source code](https://github.com/JetBrains/intellij-community/blob/573a1eebd5cb39a01a18ceadd4e4e2a7f36046bb/python/helpers/pycharm/teamcity/pytest_plugin.py#L319) PyCharm is using a plugin, and this line applies pprint to left and right unconditionally. — Coyotwill, Dec 24 '20 at 00:05

score 1 · Accepted Answer · answered Dec 24 '20 at 02:23

So it turns out this is an hard-coded behaviour of the pytest plugin used by PyCharm. The plugin always applies pprint.pformat() to the left and right values.

The behaviour described in the question then occurs when the strings are longer than 80 characters and contain white spaces.

One possible workaround is to override the plugin's pytest_assertrepr_compare hook. Here's a version that worked for me. Simply paste it in your conftest.py.

import pprint
import pytest

@pytest.hookimpl(tryfirst=True)
def pytest_assertrepr_compare(config, op, left, right):
    if op in ('==', '!='):
        return ['{0} {1} {2}'.format(pprint.pformat(left, width=999), op, pprint.pformat(right, width=999))]

Another possible hack it to monkeypatch pprint.pformat:

import pytest
import pprint
from functools import partial

@pytest.fixture(scope='function', autouse=True)
def fix_long_string_diffs(monkeypatch):
    monkeypatch.setattr(pprint, 'pformat', partial(pprint.pformat, width=999))

First one is definitely the way to go. This reminds me of https://stackoverflow.com/q/50608443/2650249; I guess I was the one proposing the assertrepr hook in the first place :-) — hoefling, Dec 24 '20 at 09:10

Annoying diff format for long strings using pytest + Pycharm

1 Answers1