How to replace pytest output for test completely with a custom string?

Question

Scenario

I am writing a package that requires me to externally call pytest from within a pytest run via subprocess. So obviously the output captured from the subprocess is exactly the error I want to display as it has all the nice formatting and info that pytest provides. Unfortunately, currently the main pytest call just shows internal code of my wrapper instead of the nice subprocess output which, after I print it, only is shown in the captured stdout section of pytest. I would like to format the output for failures and errors as if the code was called directly and hide that a subprocess call was made. Hence, I basically want to completely replace the output for one test-function with a different string. Is this possible?

MWE

Let's look at a MWE of a simple wrapped function (not doing anything useful, but the shortest MWE I could think of):

import functools
from subprocess import Popen, PIPE
import sys


def call_something(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        # for the sake of simplicity, a dummy call
        p = Popen([sys.executable, '-c', 'import numby'], stderr=PIPE, stdout=PIPE)
        # let's imagine this is the perfect string output we want
        # to print instead of the actual output generated from...
        error = p.communicate()[1].decode("utf-8")

        if p.returncode != 0:
            # ... this line
            raise AssertionError(f'{error}')

    return wrapper

@call_something
def test_something():
    assert 1 == 2

As you can see I have my test_something() function which in the subprocess will fail.

Current output

If I run pytest with this file I get:

================================== FAILURES ===================================
_______________________________ test_something ________________________________

func = <function test_something at 0x000001EA414F1400>, args = (), kwargs = {}
p = <subprocess.Popen object at 0x000001EA414A67B8>
error = 'Traceback (most recent call last):\r\n  File "<string>", line 1, in <module>\r\nModuleNotFoundError: No module named \'numby\'\r\n'

    def wrapper(*args, **kwargs):
        # for the sake of simplicity, a dummy call
        p = Popen([sys.executable, '-c', 'import numby'], stderr=PIPE, stdout=PIPE)
        # let's imagine this is the perfect string output we want
        # to print instead of the actual output generated from...
        error = p.communicate()[1].decode("utf-8")

        if p.returncode != 0:
            # ... this line
>               raise AssertionError(f'{error}')
E               AssertionError: Traceback (most recent call last):
E                 File "<string>", line 1, in <module>
E               ModuleNotFoundError: No module named 'numby'

test_me.py:18: AssertionError
========================== 1 failed in 0.18 seconds ===========================

Obviously, I don't want to show the details of the wrapper function. Instead

Desired output

I would like to show what happens in the subprocess. So it should look like this (or similar).

================================== FAILURES ===================================
_______________________________ test_something ________________________________

<string captured from subprocess>
========================== 1 failed in 0.11 seconds ===========================

So my question in smaller bits:

Question in short

• How to replace the pytest output for test with a custom string from my wrapper?

Answer 1

When raising errors, you can use the from syntax to suppress or change how exceptions are chained. For example, consider the following:

try:
    a / b
except ZeroDivisionError:
    raise ValueError('Invalid b value')

This will show a ZeroDivisionError, if b == 0 followed by a ValueError, but you may want to suppress the ZeroDivisionError, because the only relevant part is the ValueError. So you'd instead write:

try:
    a / b
except ZeroDivisionError:
    raise ValueError('Invalid b value') from None

This will only show the ValueError, telling you that b was wrong

There are other things you can do with the from syntax which will be relevant here, see this thread for details.

I do something very similar to what you're trying by suppressing weird pipe errors and instead raising ModuleNotFoundError:

# Test if <module> is callable
try:
    sp.run('<run some module>', shell=True, check=True, stdout=sp.PIPE)
except sp.CalledProcessError as exc:
    raise ModuleNotFoundError(
        '<some module> not working. Please ensure it is installed and can be called with the command: <command>'
    ) from exc

Note the use of the from exc here.