How can I get pytest to not catch exceptions

Question

When I run pytest in the vscode debugger with "Uncaught Exceptions" checked, and there are test errors, no uncaught exceptions occur, because pytest catches them do its results reporting. How can I tell pytest to just let the exceptions happen? So that I can catch them in the vscode debugger?

Basically I want behavior like --pdb but I want it to launch the vscode debugger not pdb. The flag --pdbcls sounds promising but not sure what <module>:<class> to give it.

Note: Normally I would just have it break on Raised Exceptions. But our code has tons of raised-but-caught exceptions so this option is not useful.

Here's a video of vscode not breaking when an AssertionError fires while debugging a pytest test:

@rioV8's suggestion below does break the on the exception, but for some reason there is no stack which means you can't debug from there:

I must be missing something because no one else seems to need this capability. But to me this seems like absolutely the most basic simplest thing one could do with a testing framework and a debugger: as a developer I want to debug from the point where the error is raised.

There must be some totally other way people are using a debugger with pytest, some obvious technique I'm overlooking.

Answer 1

I raised an issue with pytest and they made a suggestion that works. Add this to your conftest.py:

import os
import pytest

if os.getenv('_PYTEST_RAISE', "0") != "0":

    @pytest.hookimpl(tryfirst=True)
    def pytest_exception_interact(call):
        raise call.excinfo.value

    @pytest.hookimpl(tryfirst=True)
    def pytest_internalerror(excinfo):
        raise excinfo.value

Then in your "request": "test" launch.json you can toggle that env var:

    {
        "name": "Debug Tests",
        "type": "python",
        "request": "test",
        "console": "integratedTerminal",
        "justMyCode": true,
        "env": {
            "_PYTEST_RAISE": "1"
        },
    }

If _PYTEST_RAISE is set and you check only the Uncaught Exceptions box you will break right where the uncaught exception was raised not before.

I also opened an issue with debugpy and they had some ideas, but no solution today. The _PYTEST_RAISE trick works 100% solves this for me.

Answer 2

I've noticed a few things that might be relevant. Even though I'm not ready to dig deeper right now, I'll post them here in case they might be useful to someone.

Workaround:

A simple hack I've used is to just (temporarily) call the relevant test case directly and "Run and Debug" the file rather than using the "Debug Test Case" button:

def test_example():
    assert False


test_example()

Result:

Notes:

1. The important difference in behavior correlates with specifying "request": "test" vs specifying "request": "launch" in the launch.json configuration. (There is also a difference between the two types in whether or not the justMyCode option gets applied from a global launch configuration as described here).
2. If you have a python "request": "test" specified, then VSCode doesn't appear to let you debug the file in launch mode as I did above.
3. In my scenario, I had a test failing because of an exception I was causing in a third-party library (my fault, of course). Breakpoints in third-party code are apparently only triggered if the "justMyCode": false option is added.

launch.json:

{
    "version": "0.2.0",
    "configurations": [
        
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": false,            
        },
        // {
        //     "name": "Debug Unit Test",
        //     "type": "python",
        //     "request": "test",
        //     "console": "integratedTerminal",
        //     "justMyCode": false,
        // }        
    ]
}

Answer 3

The behavior is not identical but the test runner stops at the uncaught exception.

Add the file conftest.py to the directory with your tests.

# conftest.py
def pytest_exception_interact(node, call, report):
  print( call.excinfo.traceback[0] )
  pass

This adds a hook for any uncaught exceptions.

Place a break point at the line with pass.

Then Debug the test.

Look in the Debug console and you see the file and line number where the exception happened.

test_example.py F  File 'path\\test_example.py':4 in test_example
  assert False

And the debugger stops at the pass line.

You can make this hook function as fancy as you like.

Answer 4

This is similar to the answer above, however I prefer to have it automatically engage when debugging a test. Put the following in the conftest.py:

# conftest.py
import sys
import pytest

def is_debugging():
    if 'debugpy' in sys.modules:
        return True
    return False


# enable_stop_on_exceptions if the debugger is running during a test
if is_debugging():
    @pytest.hookimpl(tryfirst=True)
    def pytest_exception_interact(call):
        raise call.excinfo.value

    @pytest.hookimpl(tryfirst=True)
    def pytest_internalerror(excinfo):
        raise excinfo.value

What is happening is that it checks to see if "debugpy" is active. If it is, we assume we are debugging a test and stop on uncaught exceptions. Otherwise, it behaves as normal. This means if you run pytest from the command-line, the "debugpy" is not active, and it will behave as normal. One nice side effect is that this will also stop the debugger if there is any exception including import exceptions, language exceptions, etc.