Warnings from caller's perspective (aka Python-equivalent of Perl's carp)?

Question

Short version:

Is there way to achieve in Python the same effect achieved by Perl's Carp::carp utility?

Long version (for those unfamiliar with Carp::carp):

Suppose we are implementing some library API function (i.e., it is meant to be used by other programmers in their code), say spam, and suppose that spam includes some code to check the validity of the arguments passed to it. Of course, this code is supposed to raise an exception if any problem with these arguments is detected. Let's say that we want to make the associated error message and traceback as helpful as possible to someone debugging some client code.

Ideally, the last line of the traceback produced by this raised exception should pinpoint the "offending code", namely the line in the client code where spam was called with invalid arguments.

Unfortunately, this is not what would happen, at least by default, using Python. Instead, the last line of the traceback will refer to somewhere in the internals of the library code, where the exception was actually raise'd, which would be quite obscure to the intended audience of this particular traceback.

Example:

# spam.py (library code)
def spam(ham, eggs):
    '''
    Do something stupid with ham and eggs.

    At least one of ham and eggs must be True.
    '''
    _validate_spam_args(ham, eggs)
    return ham == eggs

def _validate_spam_args(ham, eggs):
    if not (ham or eggs):
        raise ValueError('if we had ham '
                         'we could have ham and eggs '
                         '(if we had eggs)')



# client.py (client code)
from spam import spam

x = spam(False, False)

When we run client.py, we get:

% python client.py
Traceback (most recent call last):
  File "client.py", line 3, in <module>
    x = spam(False, False)
  File "/home/jones/spam.py", line 7, in spam
    _validate_spam_args(ham, eggs)
  File "/home/jones/spam.py", line 12, in _validate_spam_args
    raise ValueError('if we had ham '
ValueError: if we had ham we could have ham and eggs (if we had eggs)

whereas what we want would be closer to:

% python client.py
Traceback (most recent call last):
  File "client.py", line 3, in <module>
    x = spam(False, False)
ValueError: if we had ham we could have ham and eggs (if we had eggs)

...with the offending code (x = spam(False, False)) as the last line of the traceback.

What we need is some way to report the error "from the perspective of the caller" (which is what Carp::carp lets one do in Perl).

EDIT: Just to be clear, this question is not about LBYL vs EAFP, nor about preconditions or programming-by-contract. I am sorry if I gave this wrong impression. This question is about how to produce a traceback starting from a few (one, two) levels up the call stack.

EDIT2: Python's traceback module is an obvious place to look for a Python-equivalent of Perl's Carp::carp, but after studying it for some time I was not able to find any way to use it for what I want to do. FWIW, Perl's Carp::carp allows fine-adjusting of the initial frame for the traceback by exposing the global (hence dynamically scoped) variable $Carp::CarpLevel. Non-API library functions that may carp-out, local-ize and increase this variable on entry (e.g. local $Carp::CarpLevel += 1;). I don't see anything even remotely like this Python's traceback module. So, unless I missed something, any solution that uses Python's traceback would have to take a rather different tack...

Answer 1

This is really just a matter of convention, exception handling in python is designed to be used heavily (beg forgiveness rather than ask permission). And given that you're working in a different language space, you want to follow those conventions - ie/ you really do want to let developers know where the site of the exception was. But if your really do need to do this ...

Using the Inspect Module

The inspect module will do pretty much everything you need to reconstruct a nice version of carp, that works without having to worry about decorators (see below). As per the comments in this answer, it may be that this approach will break on pythons other than cpython

# revised carp.py
import sys
import inspect

def carp( msg ):
    # grab the current call stack, and remove the stuff we don't want
    stack = inspect.stack()
    stack = stack[1:]

    caller_func = stack[0][1]
    caller_line = stack[0][2]
    sys.stderr.write('%s at %s line %d\n' % (msg, caller_func, caller_line))

    for idx, frame in enumerate(stack[1:]):
        # The frame, one up from `frame`
        upframe = stack[idx]
        upframe_record = upframe[0]
        upframe_func   = upframe[3]
        upframe_module = inspect.getmodule(upframe_record).__name__

        # The stuff we need from the current frame
        frame_file = frame[1]
        frame_line = frame[2]

        sys.stderr.write( '\t%s.%s ' % (upframe_module, upframe_func) )
        sys.stderr.write( 'called at %s line %d\n' % (frame_file, frame_line) )

    # Exit, circumventing (most) exception handling
    sys.exit(1)

Which for the following example:

  1 import carp
  2
  3 def f():
  4     carp.carp( 'carpmsg' )
  5
  6 def g():
  7     f()
  8
  9 g()

Produces the output:

msg at main.py line 4
        __main__.f called at main.py line 7
        __main__.g called at main.py line 9

Using Traceback

This was the original approach proposed.

An equivalent to carp could also be written in python by manipulating traceback objects, see the documentation in the traceback module. The main challenge in doing this turns out to be injecting the exception and traceback print code. It is worth noting that the code in this section is very fragile.

# carp.py
import sys
import traceback

'''
carp.py - partial emulation of the concept of perls Carp::carp
'''

class CarpError(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)

def carpmain( fun ):
    def impl():
        try:
            fun()
        except CarpError as ex:
            _, _, tb = sys.exc_info()
            items = traceback.extract_tb(tb)[:-1]
            filename, lineno, funcname, line = items[-1]
            print '%s at %s line %d' % (ex.value, filename, lineno)
            for item in items[1:]:
                filename, lineno, funcname, line = item
                print '\t%s called at %s line %d' % (funcname, filename, lineno)
    return impl

def carp( value ):
    raise CarpError( value )

Which can be called using the following basic process:

import carp

def g():
    carp.carp( 'pmsg' )

def f():
    g()

@carp.carpmain
def main():
    f()

main()

The output of which is:

msg at foo.py line 4
    main called at foo.py line 12
    f called at foo.py line 7
    g called at foo.py line 4

Perl Reference Example

For completeness, both solutions proposed in this answer were debugged by comparing results to this equivalent perl example:

  1 use strict;
  2 use warnings;
  3 use Carp;
  4
  5 sub f {
  6     Carp::carp("msg");
  7 }
  8
  9 sub g {
 10     f();
 11 }
 12
 13 g();

Which has the output:

msg at foo.pl line 6
    main::f() called at foo.pl line 10
    main::g() called at foo.pl line 13

Answer 2

You could use try..except in the top level API function (foo) to raise a different exception:

class FooError(Exception): pass

def foo():
    try:
        bar()
    except ZeroDivisionError:
        raise FooError()

def bar():
    baz()

def baz():
    1/0

foo()

Thus, when a API user calls foo and an exception is raised, all they see is the FooError and not the internal ZeroDivisionError.

Answer 3

What you want to do is known as establishing function preconditions, and there is no language support for it in Python. Python is also not as thoroughly hackable as perl (unless perhaps if you are using PyPy) so it can't be added in an entirely seamless way.

That being said, the module PyContracts seems to do this relatively smoothly using function decorators and string-based precondition specs. I haven't used the module myself, but it does seem like it might get you something closer what you want. Here is the first example given on its info page:

@contract
def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]':
     # Requires b to be a nonempty list, and the return
     # value to have the same length.
     ...