How to automatically switch to debug mode on a warning?

Question

I want to debug a warning that occurs well into the code's execution.

A simple breakpoint won't do, because the line that results in the warning executes millions of times without warning before the first warning occurs.

Also, the line where this is happening is in library code (more precisely, in pandas/core/common.py), so my preference is not to modify the code at all.

I just want to stop the program's execution right when it emits the warning, and inspect the stack at this point, either with pdb or with ipdb.

Is there a way to configure either debugger to automatically enter single-step mode upon the issuing of a warning?

Do you mean basically what happens if you insert `import pdb;pdb.set_trace()` at that point? — Two-Bit Alchemist, Dec 21 '15 at 18:40
@Two-BitAlchemist: yes, pretty much, except that, as I wrote, I don't want to actually modify the code being debugged. — kjo, Dec 21 '15 at 18:50
A warning is a warning, not an exception. The debugger will not pick it up. Usually you either change the code to prevent the warning or your suppress it as described here: https://docs.python.org/2/library/warnings.html . If you need help with a particular warning, just ask! — Klaus D., Dec 21 '15 at 18:55

user2683246 · Accepted Answer · 2016-01-18T21:23:01.373

You can write a script dbg.py:

import pdb, warnings, sys
import __builtin__

if __name__ == '__main__':
    args, n = [], len(sys.argv)
    if n < 2:
        sys.exit(1)
    elif n > 2:
        args.append(__builtin__.__dict__[sys.argv[2]])
        if n > 3:
            args.append(int(sys.argv[3]))
    warnings.simplefilter('error', *args)  # treat warnings as exceptions
    try:
        execfile(sys.argv[1])
    except:
        pdb.post_mortem(sys.exc_info()[-1])

You can then use it to debug your script like that. Pass in your script name as the first argument if you want to run pdb on any warning:

$ python dbg.py yourscript.py

Pass in warning type as the second argument if you only want pdb to run when some particular type of warning is raised:

$ python dbg.py yourscript.py DeprecationWarning

Line number as the third argument:

$ python dbg.py yourscript.py DeprecationWarning 342

You can also rewrite the code using warnings.filterwarnings instead of warnings.simplefilter to make warnings filtering even more flexible.

If you try to use this you should be aware that in Python 3 the module `__builtin__` was renamed `builtins`, more info: https://stackoverflow.com/questions/9047745/where-is-the-builtin-module-in-python3-why-was-it-renamed — Heberto Mayorquin, Aug 05 '18 at 03:15

score 7 · Answer 2 · answered Sep 21 '19 at 16:11

I found the answer offered by @user2683246 elegant and useful. Here is a variant of the solution modified for compatibility with Python3 (tested with Python 3.7):

#!/usr/bin/env python

import pdb, warnings, sys
import builtins

if __name__ == '__main__':
    args, n = [], len(sys.argv)
    if n < 2:
        sys.exit(1)
    elif n > 2:
        args.append(builtins.__dict__[sys.argv[2]])
        if n > 3:
            args.append(int(sys.argv[3]))
    warnings.simplefilter('error', *args)  # treat warnings as exceptions
    try:
        with open(sys.argv[1]) as f:
            code = compile(f.read(), sys.argv[1], 'exec')
            exec(code)
    except:
        pdb.post_mortem(sys.exc_info()[-1])

Notable changes:

Replace the execfile() call with the Python 3 variant; and
Replace __builtin__ with builtins.

I'm trying to use this with the Pint library, which defines its own warning classes in pint.error. But when I use names like UnitStrippedWarning I get a KeyError. How do I tell dbg.py to look at names in the UnitStrippedWarning class that pint defines? — Michael Tiemann, Oct 21 '22 at 11:40

score 3 · Answer 3 · answered Nov 06 '18 at 00:41

Instead of treating the warning as error, warnings.catch_warnings link can be used to access warning list and switch execution into PDB session if the scope of code which throws warning can be identified.

However I recommand to start your program with PDB, set a breakpoint to break the execution when detecting warning number change of warning list after the problematic code snippet. You will be benefited if your debugging code snippet is in a loop.

example:

import warnings

with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter('always')
    warningNum = len(w)
    for i in range(someNumber):
        "your code probably throw warning"

        if len(w) != warningNum:
            warningNum = len(w) #set break point here

run script with python -m pdb yourscript.py and set a breakpoint in line warningNum = len(w), then execution can be suspended when detecting warning number change.

score 2 · Answer 4 · answered Dec 21 '15 at 19:23

https://pypi.python.org/pypi/rpdb/

i found rpdb very handy to debug such problem, when you have no control of starting the program. what you need is temporarily modify pandas/core/common.py to add

import rpdb
debugger = rpdb.Rpdb(port=12345)
debugger.set_trace()

when the warning is triggered, the debugger will be there waiting for connection. you then connect to the debugger and inspect the stack.

How to automatically switch to debug mode on a warning?

4 Answers4