14

In the module warnings (https://docs.python.org/3.5/library/warnings.html) there is the ability to raise a warning that appears to come from somewhere earlier in the stack:

warnings.warn('This is a test', stacklevel=2)

Is there an equivalent for raising errors? I know I can raise an error with an alternative traceback, but I can't create that traceback within the module since it needs to come from earlier. I imagine something like:

tb = magic_create_traceback_right_here()
raise ValueError('This is a test').with_traceback(tb.tb_next)

The reason is that I am developing a module that has a function module.check_raise that I want to raise an error that appears to originate from where the function is called. If I raise an error within the module.check_raise function, it appears to originate from within module.check_raise, which is undesired.

Also, I've tried tricks like raising a dummy exception, catching it, and passing the traceback along, but somehow the tb_next becomes None. I'm out of ideas.

Edit:

I would like the output of this minimal example (called tb2.py):

import check_raise

check_raise.raise_if_string_is_true('True')

to be only this:

Traceback (most recent call last):
  File "tb2.py", line 10, in <module>
    check_raise.raise_if_string_is_true(string)
RuntimeError: An exception was raised.
Joel
  • 2,065
  • 2
  • 19
  • 30
  • Hm, if you use `filter` to turn the warning into an error and then just call `warnings.warn` will that do the trick? – Dimitris Fasarakis Hilliard Dec 09 '15 at 10:25
  • Judging from the [`source`](https://hg.python.org/cpython/file/3.5/Lib/warnings.py) it will `raise` it, I'm just not sure if it will do it in the way you wish (and looking at it, I'm thinking it won't). – Dimitris Fasarakis Hilliard Dec 09 '15 at 10:27
  • @Jim I tried this, it raises an error, but the traceback it prints is just the same as if you raised an error at that location: the stacklevel= argument does nothing. – Joel Dec 09 '15 at 20:18
  • Jinja2 (a templating engine) uses a lot of dirty tricks in order to accomplish something similar, but it takes a huge amount of code to implement. See https://github.com/mitsuhiko/jinja2/blob/master/jinja2/debug.py for the grubby details, and then consider whether you **really** want to go down this road... – kiwidrew Dec 10 '15 at 09:54
  • @kiwidrew Yeah, that looks terrible. I have no idea if it's even possible. – Joel Dec 10 '15 at 22:24
  • I don't think there is a way to do this, AFAIK. However, you could duck punch everything that deals with tracebacks in any significant way. For instance, make a subclass of the `logging.Logger` class that modifies the traceback (or makes a modified copy) before passing it to its `super` class. Also, you can swap out the function [`sys.excepthook`](https://docs.python.org/3.5/library/sys.html#sys.excepthook) that prints out uncaught exceptions and do the same tricks. Unless you are actually using the tracebacks for anything other than printing, this might fit your needs well enough. – eestrada Jan 15 '16 at 23:47
  • @eestrada I do want it for more than just printing unfortunately. I basically want to create my own function that acts exactly like the `raise` statement, but has different logic. I think that's probably an exercise in futility, because of how similar it is to actually just creating a statement, which would require recompiling python, which is definitely NOT the route I'm going with a package I want to distribute. http://stackoverflow.com/questions/214881/can-you-add-new-statements-to-pythons-syntax – Joel Jan 16 '16 at 00:01
  • @Joel This is where something like [Lisp macros would fit the bill nicely](http://stackoverflow.com/questions/267862/what-makes-lisp-macros-so-special#4621882) :\ . You could try this [Python macro package](https://github.com/lihaoyi/macropy) to see if it fits the bill. Caveat: I've never tried it out and it seems rather old at this point. – eestrada Jan 16 '16 at 00:08
  • Would it be possible to just `catch` the original exception and throw a new one at the appropriate point? Alternatively, can you create a tb at the appropriate point *regardless* of whether an error occurs, and pass it down the call stack to throw later if necessary? – Kyle Strand Jan 21 '16 at 18:06
  • Interesting: https://bugs.python.org/issue41399 – Martin Thoma Mar 04 '23 at 07:57

4 Answers4

3

I can't believe I am posting this

By doing this you are going against the zen.

Special cases aren't special enough to break the rules.

But if you insist here is your magical code.

check_raise.py

import sys
import traceback

def raise_if_string_is_true(string):
    if string == 'true':
        #the frame that called this one
        f = sys._getframe().f_back
        #the most USELESS error message ever
        e = RuntimeError("An exception was raised.")

        #the first line of an error message
        print('Traceback (most recent call last):',file=sys.stderr)
        #the stack information, from f and above
        traceback.print_stack(f)
        #the last line of the error
        print(*traceback.format_exception_only(type(e),e),
              file=sys.stderr, sep="",end="")

        #exit the program
        #if something catches this you will cause so much confusion
        raise SystemExit(1)
        # SystemExit is the only exception that doesn't trigger an error message by default.

This is pure python, does not interfere with sys.excepthook and even in a try block it is not caught with except Exception: although it is caught with except:

test.py

import check_raise

check_raise.raise_if_string_is_true("true")
print("this should never be printed")

will give you the (horribly uninformative and extremely forged) traceback message you desire.

Tadhgs-MacBook-Pro:Documents Tadhg$ python3 test.py
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    check_raise.raise_if_string_is_true("true")
RuntimeError: An exception was raised.
Tadhgs-MacBook-Pro:Documents Tadhg$
Tadhg McDonald-Jensen
  • 20,699
  • 5
  • 35
  • 59
  • 1
    I was running cleanup and found this. FYI, your `sys._getframe()` call is documented as internal and specialized, and currently exists in CPython and may not exist in other implementations. I believe the implication is that this function is not required by the `sys` module specification. In that sense, it's not really pure Python. – Joel Jul 24 '19 at 21:25
  • 1
    But- I believe the basic thought saying, "Don't do this" is probably the most correct answer here. – Joel Jul 24 '19 at 21:26
2

If I understand correctly, you would like the output of this minimal example:

def check_raise(function):
    try:
        return function()
    except Exception:
        raise RuntimeError('An exception was raised.')

def function():
    1/0

check_raise(function)

to be only this:

Traceback (most recent call last):
  File "tb2.py", line 10, in <module>
    check_raise(function)
RuntimeError: An exception was raised.

In fact, it's a lot more output; there is exception chaining, which could be dealt with by handling the RuntimeError immediately, removing its __context__, and re-raising it, and there is another line of traceback for the RuntimeError itself:

  File "tb2.py", line 5, in check_raise
    raise RuntimeError('An exception was raised.')

As far as I can tell, it is not possible for pure Python code to substitute the traceback of an exception after it was raised; the interpreter has control of adding to it but it only exposes the current traceback whenever the exception is handled. There is no API (not even when using tracing functions) for passing your own traceback to the interpreter, and traceback objects are immutable (this is what's tackled by that Jinja hack involving C-level stuff).

So further assuming that you're interested in the shortened traceback not for further programmatic use but only for user-friendly output, your best bet will be an excepthook that controls how the traceback is printed to the console. For determining where to stop printing, a special local variable could be used (this is a bit more robust than limiting the traceback to its length minus 1 or such). This example requires Python 3.5 (for traceback.walk_tb):

import sys
import traceback

def check_raise(function):
    __exclude_from_traceback_from_here__ = True
    try:
        return function()
    except Exception:
        raise RuntimeError('An exception was raised.')

def print_traceback(exc_type, exc_value, tb):
    for i, (frame, lineno) in enumerate(traceback.walk_tb(tb)):
        if '__exclude_from_traceback_from_here__' in frame.f_code.co_varnames:
            limit = i
            break
    else:
        limit = None
    traceback.print_exception(
        exc_type, exc_value, tb, limit=limit, chain=False)

sys.excepthook = print_traceback

def function():
    1/0

check_raise(function)

This is the output now:

Traceback (most recent call last):
  File "tb2.py", line 26, in <module>
    check_raise(function)
RuntimeError: An exception was raised.
Thomas Lotze
  • 5,153
  • 1
  • 16
  • 16
  • This is quite good. Is it possible to keep the sys.excepthook line inside of a module? I want to stick all that code in a module so it looks different. I'll update my question to be more clear. – Joel Feb 22 '16 at 02:00
  • Yes, that's technically possible. However, I'd suggest putting it in an initialisation function inside your module and call once at the beginning of your program which uses the module. By requiring the programmer to consciously initialise the hook, it becomes less megaic ("explicit is better than implicit") and, more importantly, allows doing this kind of things at process initialisation and in the correct order in case that other modules fiddle with the same facilities. Otherwise, the excepthook would be installed whenever the module happens to be imported first. – Thomas Lotze Feb 22 '16 at 08:27
  • Ah dang... that might be an issue huh? I don't want it to completely commandeer the exception system... Maybe it could do a test to make sure it's only "fixing" the exceptions generated by my module and let the others pass through unmolested...? – Joel Feb 22 '16 at 21:33
  • That's easy. If the marker name isn't found in any frame of the traceback, it is printed unchanged already. The only thing to take care of is to leave chaining intact if the traceback isn't yours, i.e. pass `chain=False` to `print_exception` only for your exceptions. – Thomas Lotze Feb 22 '16 at 21:38
0

EDIT: The previous version did not provide quotes or explanations.

I suggest referring to PEP 3134 which states in the Motivation:

Sometimes it can be useful for an exception handler to intentionally re-raise an exception, either to provide extra information or to translate an exception to another type. The __cause__ attribute provides an explicit way to record the direct cause of an exception.

When an Exception is raised with a __cause__ attribute the traceback message takes the form of:

Traceback (most recent call last):
 <CAUSE TRACEBACK>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  <MAIN TRACEBACK>

To my understanding this is exactly what you are trying to accomplish; clearly indicate that the reason for the error is not your module but somewhere else. If you are instead trying to omit information to the traceback like your edit suggests then the rest of this answer won't do you any good.


Just a note on syntax:

The __cause__ attribute on exception objects is always initialized to None. It is set by a new form of the 'raise' statement:

   raise EXCEPTION from CAUSE

which is equivalent to:

    exc = EXCEPTION
    exc.__cause__ = CAUSE
    raise exc

so the bare minimum example would be something like this:

def function():
    int("fail")

def check_raise(function):
    try:
        function()
    except Exception as original_error:
        err = RuntimeError("An exception was raised.")
        raise err from original_error

check_raise(function)

which gives an error message like this:

Traceback (most recent call last):
  File "/PATH/test.py", line 7, in check_raise
    function()
  File "/PATH/test.py", line 3, in function
    int("fail")
ValueError: invalid literal for int() with base 10: 'fail'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/PATH/test.py", line 12, in <module>
    check_raise(function)
  File "/PATH/test.py", line 10, in check_raise
    raise err from original_error
RuntimeError: An exception was raised.

However the first line of the cause is the statement in the try block of check_raise:

  File "/PATH/test.py", line 7, in check_raise
    function()

so before raising err it may (or may not) be desirable to remove the outer most traceback frame from original_error:

except Exception as original_error:
    err = RuntimeError("An exception was raised.")
    original_error.__traceback__ = original_error.__traceback__.tb_next
    raise err from original_error

This way the only line in the traceback that appears to come from check_raise is the very last raise statement which cannot be omitted with pure python code although depending on how informative the message is you can make it very clear that your module was not the cause of the problem:

err = RuntimeError("""{0.__qualname__} encountered an error during call to {1.__module__}.{1.__name__}
the traceback for the error is shown above.""".format(function,check_raise))

The advantage to raising exception like this is that the original Traceback message is not lost when the new error is raised, which means that a very complex series of exceptions can be raised and python will still display all the relevant information correctly:

def check_raise(function):
    try:
        function()
    except Exception as original_error:
        err = RuntimeError("""{0.__qualname__} encountered an error during call to {1.__module__}.{1.__name__}
the traceback for the error is shown above.""".format(function,check_raise))
        original_error.__traceback__ = original_error.__traceback__.tb_next
        raise err from original_error

def test_chain():
    check_raise(test)

def test():
    raise ValueError

check_raise(test_chain)

gives me the following error message:

Traceback (most recent call last):
  File "/Users/Tadhg/Documents/test.py", line 16, in test
    raise ValueError
ValueError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/Tadhg/Documents/test.py", line 13, in test_chain
    check_raise(test)
  File "/Users/Tadhg/Documents/test.py", line 10, in check_raise
    raise err from original_error
RuntimeError: test encountered an error during call to __main__.check_raise
the traceback for the error is shown above.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/Tadhg/Documents/test.py", line 18, in <module>
    check_raise(test_chain)
  File "/Users/Tadhg/Documents/test.py", line 10, in check_raise
    raise err from original_error
RuntimeError: test_chain encountered an error during call to __main__.check_raise
the traceback for the error is shown above.

Yes it is long but it is significantly more informative then:

Traceback (most recent call last):
  File "/Users/Tadhg/Documents/test.py", line 18, in <module>
    check_raise(test_chain)
RuntimeError: An exception was raised.

not to mention that the original error is still usable even if the program doesn't end:

import traceback

def check_raise(function):
    ...

def fail():
    raise ValueError

try:
    check_raise(fail)
except RuntimeError as e:
    cause = e.__cause__
    print("check_raise failed because of this error:")
    traceback.print_exception(type(cause), cause, cause.__traceback__)

print("and the program continues...")
Tadhg McDonald-Jensen
  • 20,699
  • 5
  • 35
  • 59
  • Close- I actually don't want the last part, the "process being checked raised and Exception". – Joel Feb 22 '16 at 01:56
  • I assume you meant you don't want the `raise x from e` to be present but I don't see why that would be an issue if part of the error specifically says `The above exception was the direct cause of the following exception:` anyway I edited the answer to explain the answer and show benefits of using it. – Tadhg McDonald-Jensen Feb 22 '16 at 20:50
  • It's just messy is all. The real motivation behind all of this is that I want this to version-check a git module. I write modules that are used by scripts. But my modules sometimes change rather rapidly- they are unstable. So I made another module (VCheck) that checks the git version of these modules. If it's the right version, then the script runs. If it isn't, then VCheck should raise an exception. I want the exception to be "the version is wrong" and traceback to the script, not "VCheck determined in its bowels that the version is wrong" and traceback to VCheck. – Joel Feb 22 '16 at 21:31
  • I have noticed that built in functions like `int` or `open` operate the way you suggest so maybe you just need to make your function in C :P – Tadhg McDonald-Jensen Feb 22 '16 at 21:31
  • Yeah, I thought about that... This is getting ridiculously complicated though. I think I'm trying to do something Python is not intended to do... – Joel Feb 22 '16 at 21:34
  • Have you considered just returning `True` if the check worked and `False` if it failed? then you could just `assert` the check in the script and that is where the last line of the traceback would point. And yeah, Trying to make the Traceback less informative then possible is very un-Pythonesque. – Tadhg McDonald-Jensen Feb 22 '16 at 21:48
0

I understand 'Don't do this'. On the other hand, there may be some special use cases i believe. I'm generating own errors (just deleting some defined frames...) this way

def get_traceback_with_removed_frames_by_line_string(lines):
    """In traceback call stack, it is possible to remove particular level defined by some line content.

    Args:
        lines (list): Line in call stack that we want to hide.

    Returns:
        string: String traceback ready to be printed.
    """
    exc = trcbck.TracebackException(*sys.exc_info())
    for i in exc.stack[:]:
        if i.line in lines:
            exc.stack.remove(i)

    return "".join(exc.format())

I return just string.

If there is concrete function that is raising, you can add it to ignored frames.

Though have in mind, that if you hide something, somebody may not understand why is something happening...

My use case was to hide only first level - decorator from my library that was decorating all user functions in framework, so error from user side was on level 1.

Daniel Malachov
  • 1,604
  • 1
  • 10
  • 13