py.test: hide stacktrace lines from unittest module

Question

py.test stacktraces look like this at the moment:

Traceback (most recent call last):
  File "/home/foo_tbz_di476/src/djangotools/djangotools/tests/ReadonlyModelTestCommon.py", line 788, in test_stale_or_missing_content_types
    self.assertEqual([], errors, 'Stale/Missing ContentTypes: %s' % '\n'.join(errors))
  File "/usr/lib64/python2.7/unittest/case.py", line 511, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python2.7/unittest/case.py", line 740, in assertListEqual
    self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib64/python2.7/unittest/case.py", line 722, in assertSequenceEqual
    self.fail(msg)
  File "/usr/lib64/python2.7/unittest/case.py", line 408, in fail
    raise self.failureException(msg)

It would be much easier for my human eyes if the output would skip the lines from the unittest module.

Example:

Traceback (most recent call last):
  File "/home/foo_tbz_di476/src/djangotools/djangotools/tests/ReadonlyModelTestCommon.py", line 788, in test_stale_or_missing_content_types
    self.assertEqual([], errors, 'Stale/Missing ContentTypes: %s' % '\n'.join(errors))

I tried option --tb=short but this does not do this.

Update

A solution without a unix pipe (like py.test ...| grep) is preferred.

Update after 8 years

I think it is time to leave ascii-universe and enter HTML. Wouldn't it be nice to have a traceback which can be collapsed/expanded?

Can you post an example test that produces this output, along with command you're using to run it. AFAIK, py.test shouldn't be outputting tracebacks like that at all. — Jamie Cockburn, Jul 07 '14 at 11:30
I've added this as a feature request on the py.test github page: https://github.com/pytest-dev/pytest/issues/1434 — dbn, Mar 04 '16 at 08:31

score 12 · Accepted Answer · edited May 23 '17 at 11:48

It looks like you are invoking pytest like:

py.test --tb=native

This form will output a python stdlib stacktrace derived from traceback.format_exception

With pytest you can add a conftest.py file(s) to your project. Here you can add any bits of code to modify pytest behaviour.

Beware! In both the following approaches use monkey patching which people may consider evil.

Option 1: String matching

This is the simplest approach but could be a problem if the string you are searching for appears in a line that you don't want to hide.

This approach patches the ReprEntryNative class in the py package which is a dependancy of pytest.

Put the following code in you conftest.py

import py

def test_skip_line(line):
    """decide which lines to skip, the code below will also skip the next line if this returns true"""
    return 'unittest' in line

class PatchedReprEntryNative(py._code.code.ReprEntryNative):
    def __init__(self, tblines):
        self.lines = []
        while len(tblines) > 0:
            line = tblines.pop(0)
            if test_skip_line(line):
                # skip this line and the next
                tblines.pop(0)
            else:
                self.lines.append(line)
py._code.code.ReprEntryNative = PatchedReprEntryNative

Option 2: traceback frame inspection

If string matching isnt true enough for you, we can inspect the traceback before it gets dumped to strings and only output frames that aren't from a set of modules.

This approach patches the traceback.extract_tb function which probably kills puppies.

Put the following code in you conftest.py

import inspect
import linecache
import traceback
import unittest.case
import sys    

SKIPPED_MODULES = [
    unittest.case
]

def test_skip_frame(frame):
    module = inspect.getmodule(frame)
    return module in SKIPPED_MODULES

def tb_skipper(tb):
    tbnext = tb.tb_next
    while tbnext is not None:
        if test_skip_frame(tbnext.tb_frame):
            tbnext = tbnext.tb_next
        else:
            yield tbnext
    yield None

def new_extract_tb(tb, limit = None):
    if limit is None:
        if hasattr(sys, 'tracebacklimit'):
            limit = sys.tracebacklimit
    list = []
    n = 0
    new_tb_order = tb_skipper(tb) # <-- this line added
    while tb is not None and (limit is None or n < limit):
        f = tb.tb_frame
        lineno = tb.tb_lineno
        co = f.f_code
        filename = co.co_filename
        name = co.co_name
        linecache.checkcache(filename)
        line = linecache.getline(filename, lineno, f.f_globals)
        if line: line = line.strip()
        else: line = None
        list.append((filename, lineno, name, line))
        tb = next(new_tb_order) # <-- this line modified, was tb = tb.tb_next
        n = n+1
    return list
traceback.extract_tb = new_extract_tb

I think the hip thing to do is to skip modules with __unittest =True — dbn, Mar 04 '16 at 08:31
Thanks much for the quality post! Would've commented my update but didn't have enough reputation to do so :( . You'll see my update of option 1 below. — neozen, Apr 10 '21 at 04:39

score 2 · Answer 2 · answered Jun 25 '14 at 07:28

2

Try piping the output to grep with an inverted pattern. That will print all the lines except those that match the pattern.

python all_tests.py | grep -v "usr/lib64/python2.7/unittest"

answered Jun 25 '14 at 07:28

Mattias Backman

927
12
25

Every stacktrace frame gets printed with two lines: first the source file name, and the next file shows the source code. Your solutions only filters out the lines which show the source file name. The output looks strange. – guettli Jun 25 '14 at 08:05
Add -A 1 to the command line. That includes one line after the match. I hope that works as intended with a reverse filter. – Mattias Backman Jun 25 '14 at 08:32

score 1 · Answer 3 · answered Jul 08 '14 at 19:44

If you know what os your using and don't care for the directory splits, you can remove the import os and replace os.sep with the appropriate seperator

This will remove all traceback with wherever the unittest module is and the line after it.

import os
import sys
import unittest

class Stderr(object):
    def __init__(self):
        self.unittest_location = (os.sep).join(unittest.__file__.split(os.sep)[:-1])
        self.stderr = sys.__stderr__
        self.skip = False

    def write(self, text):
        if self.skip and text.find("\n") != -1: self.skip=False
        elif self.skip: pass
        else:
            self.skip = text.find(self.unittest_location) != -1
            if not self.skip: self.stderr.write(text)

sys.stderr = Stderr()

neozen · Answer 4 · 2021-12-01T23:37:20.143

All credit to https://stackoverflow.com/a/24679193/59412 I just took the liberty of porting it up to the latest pytest (6.2.3 as of writing). Stick this in your conftest.py. (won't work for unittest code though... since that's not in site-packages)

Option 1:

(filters all pip-installed packages by default)

# 5c4048fc-ccf1-44ab-a683-78a29c1a98a6
import _pytest._code.code
def should_skip_line(line):
    """
    decide which lines to skip
    """
    return 'site-packages' in line

class PatchedReprEntryNative(_pytest._code.code.ReprEntryNative):
    def __init__(self, tblines):
        self.lines = []
        while len(tblines) > 0:
            # [...yourfilter...]/test_thing.py", line 1, in test_thing
            line = tblines.pop(0)
            if should_skip_line(line):
                # some line of framework code you don't want to see either...
                tblines.pop(0)
            else:
                self.lines.append(line)
_pytest._code.code.ReprEntryNative = PatchedReprEntryNative
del _pytest._code.code

Only works with --tb=native. Just like the answer I ported up to this version.

Option 2:

TBD

Gaëtan Lehmann · Answer 5 · 2022-08-22T13:49:36.397

Another option that monkey patch pytest, but works without the --tb=native option. Just put this code in your contest.py:

import pathlib
from _pytest._code import code
from _pytest._code.code import ReprTraceback

def ishidden(self) -> bool:
    return self._ishidden() or 'site-packages' in pathlib.Path(self.path).parts
code.TracebackEntry._ishidden = code.TracebackEntry.ishidden
code.TracebackEntry.ishidden = ishidden

def repr_traceback(self, excinfo: code.ExceptionInfo[BaseException]) -> "ReprTraceback":
    traceback = excinfo.traceback
    if True:  # self.tbfilter:  <- filtering was not done for nested exception, so force it
        traceback = traceback.filter()
    # make sure we don't get an empty traceback list
    if len(traceback) == 0:
        traceback.append(excinfo.traceback[-1])

    if isinstance(excinfo.value, RecursionError):
        traceback, extraline = self._truncate_recursive_traceback(traceback)
    else:
        extraline = None

    last = traceback[-1]
    entries = []
    if self.style == "value":
        reprentry = self.repr_traceback_entry(last, excinfo)
        entries.append(reprentry)
        return ReprTraceback(entries, None, style=self.style)

    for index, entry in enumerate(traceback):
        einfo = (last == entry) and excinfo or None
        reprentry = self.repr_traceback_entry(entry, einfo)
        entries.append(reprentry)
    return ReprTraceback(entries, extraline, style=self.style)
code.FormattedExcinfo.repr_traceback = repr_traceback

del code

We get a much better output when using external libraries like playwright, making the default traceback format actually usable:

__________________________________________________________________ test_authenticated_access_to_the_application ___________________________________________________________________

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
        with page.expect_navigation(wait_until='networkidle', timeout=5000):
>           page.goto(base_url)
E           playwright._impl._api_types.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL
E           =========================== logs ===========================
E           navigating to "", waiting until "load"
E           ============================================================

tests/gui/step_defs/star_steps.py:16: Error

During handling of the above exception, another exception occurred:

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
>       with page.expect_navigation(wait_until='networkidle', timeout=5000):
E       playwright._impl._api_types.TimeoutError: Timeout 5000ms exceeded.
E       =========================== logs ===========================
E       waiting for navigation until 'networkidle'
E       ============================================================

tests/gui/step_defs/star_steps.py:15: TimeoutError

compared to the unpatched version:

__________________________________________________________________ test_authenticated_access_to_the_application ___________________________________________________________________

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
        with page.expect_navigation(wait_until='networkidle', timeout=5000):
>           page.goto(base_url)

tests/gui/step_defs/star_steps.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Page url='about:blank'>, url = ''

    def goto(
        self,
        url: str,
        *,
        timeout: float = None,
        wait_until: Literal["commit", "domcontentloaded", "load", "networkidle"] = None,
        referer: str = None
    ) -> typing.Optional["Response"]:
        """Page.goto
    
        Returns the main resource response. In case of multiple redirects, the navigation will resolve with the first
        non-redirect response.
    
        The method will throw an error if:
        - there's an SSL error (e.g. in case of self-signed certificates).
        - target URL is invalid.
        - the `timeout` is exceeded during navigation.
        - the remote server does not respond or is unreachable.
        - the main resource failed to load.
    
        The method will not throw an error when any valid HTTP status code is returned by the remote server, including 404 \"Not
        Found\" and 500 \"Internal Server Error\".  The status code for such responses can be retrieved by calling
        `response.status()`.
    
        > NOTE: The method either throws an error or returns a main resource response. The only exceptions are navigation to
        `about:blank` or navigation to the same URL with a different hash, which would succeed and return `null`.
        > NOTE: Headless mode doesn't support navigation to a PDF document. See the
        [upstream issue](https://bugs.chromium.org/p/chromium/issues/detail?id=761295).
    
        Shortcut for main frame's `frame.goto()`
    
        Parameters
        ----------
        url : str
            URL to navigate page to. The url should include scheme, e.g. `https://`. When a `baseURL` via the context options was
            provided and the passed URL is a path, it gets merged via the
            [`new URL()`](https://developer.mozilla.org/en-US/docs/Web/API/URL/URL) constructor.
        timeout : Union[float, NoneType]
            Maximum operation time in milliseconds, defaults to 30 seconds, pass `0` to disable timeout. The default value can be
            changed by using the `browser_context.set_default_navigation_timeout()`,
            `browser_context.set_default_timeout()`, `page.set_default_navigation_timeout()` or
            `page.set_default_timeout()` methods.
        wait_until : Union["commit", "domcontentloaded", "load", "networkidle", NoneType]
            When to consider operation succeeded, defaults to `load`. Events can be either:
            - `'domcontentloaded'` - consider operation to be finished when the `DOMContentLoaded` event is fired.
            - `'load'` - consider operation to be finished when the `load` event is fired.
            - `'networkidle'` - consider operation to be finished when there are no network connections for at least `500` ms.
            - `'commit'` - consider operation to be finished when network response is received and the document started loading.
        referer : Union[str, NoneType]
            Referer header value. If provided it will take preference over the referer header value set by
            `page.set_extra_http_headers()`.
    
        Returns
        -------
        Union[Response, NoneType]
        """
    
        return mapping.from_impl_nullable(
>           self._sync(
                self._impl_obj.goto(
                    url=url, timeout=timeout, waitUntil=wait_until, referer=referer
                )
            )
        )

.venv/lib/python3.10/site-packages/playwright/sync_api/_generated.py:7285: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Page url='about:blank'>, coro = <coroutine object Page.goto at 0x7f3b6b5e1700>

    def _sync(self, coro: Awaitable) -> Any:
        __tracebackhide__ = True
        g_self = greenlet.getcurrent()
        task = self._loop.create_task(coro)
        setattr(task, "__pw_stack__", inspect.stack())
        setattr(task, "__pw_stack_trace__", traceback.extract_stack())
    
        task.add_done_callback(lambda _: g_self.switch())
        while not task.done():
            self._dispatcher_fiber.switch()
        asyncio._set_running_loop(self._loop)
>       return task.result()

.venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Page url='about:blank'>, url = '', timeout = None, waitUntil = None, referer = None

    async def goto(
        self,
        url: str,
        timeout: float = None,
        waitUntil: DocumentLoadState = None,
        referer: str = None,
    ) -> Optional[Response]:
>       return await self._main_frame.goto(**locals_to_params(locals()))

.venv/lib/python3.10/site-packages/playwright/_impl/_page.py:496: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Frame name= url='about:blank'>, url = '', timeout = None, waitUntil = None, referer = None

    async def goto(
        self,
        url: str,
        timeout: float = None,
        waitUntil: DocumentLoadState = None,
        referer: str = None,
    ) -> Optional[Response]:
        return cast(
            Optional[Response],
            from_nullable_channel(
>               await self._channel.send("goto", locals_to_params(locals()))
            ),
        )

.venv/lib/python3.10/site-packages/playwright/_impl/_frame.py:136: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Channel object at 0x7f3b7095e7a0>, method = 'goto', params = {'url': ''}

    async def send(self, method: str, params: Dict = None) -> Any:
>       return await self._connection.wrap_api_call(
            lambda: self.inner_send(method, params, False)
        )

.venv/lib/python3.10/site-packages/playwright/_impl/_connection.py:43: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Connection object at 0x7f3b70976cb0>, cb = <function Channel.send.<locals>.<lambda> at 0x7f3b6b5a53f0>, is_internal = False

    async def wrap_api_call(
        self, cb: Callable[[], Any], is_internal: bool = False
    ) -> Any:
        if self._api_zone.get():
            return await cb()
        task = asyncio.current_task(self._loop)
        st: List[inspect.FrameInfo] = getattr(task, "__pw_stack__", inspect.stack())
        metadata = _extract_metadata_from_stack(st, is_internal)
        if metadata:
            self._api_zone.set(metadata)
        try:
>           return await cb()

.venv/lib/python3.10/site-packages/playwright/_impl/_connection.py:369: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <playwright._impl._connection.Channel object at 0x7f3b7095e7a0>, method = 'goto', params = {'url': ''}, return_as_dict = False

    async def inner_send(
        self, method: str, params: Optional[Dict], return_as_dict: bool
    ) -> Any:
        if params is None:
            params = {}
        callback = self._connection._send_message_to_server(self._guid, method, params)
        if self._connection._error:
            error = self._connection._error
            self._connection._error = None
            raise error
        done, _ = await asyncio.wait(
            {
                self._connection._transport.on_error_future,
                callback.future,
            },
            return_when=asyncio.FIRST_COMPLETED,
        )
        if not callback.future.done():
            callback.future.cancel()
>       result = next(iter(done)).result()
E       playwright._impl._api_types.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL
E       =========================== logs ===========================
E       navigating to "", waiting until "load"
E       ============================================================

.venv/lib/python3.10/site-packages/playwright/_impl/_connection.py:78: Error

During handling of the above exception, another exception occurred:

page = <Page url='about:blank'>, base_url = ''

    @when('I go to the application')
    def provide_creds(page: Page, base_url: str):
>       with page.expect_navigation(wait_until='networkidle', timeout=5000):

tests/gui/step_defs/star_steps.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:66: in __exit__
    self._event.value
.venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py:46: in value
    raise exception
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    async def continuation() -> Optional[Response]:
>       event = await wait_helper.result()
E       playwright._impl._api_types.TimeoutError: Timeout 5000ms exceeded.
E       =========================== logs ===========================
E       waiting for navigation until 'networkidle'
E       ============================================================

.venv/lib/python3.10/site-packages/playwright/_impl/_frame.py:197: TimeoutError

py.test: hide stacktrace lines from unittest module

5 Answers5

Option 1: String matching

Option 2: traceback frame inspection

Linked