Why can't I pickle an error's Traceback in Python?

Question

I've since found a work around, but still want to know the answer.

Answer 1

The traceback holds references to the stack frames of each function/method that was called on the current thread, from the topmost-frame on down to the point where the error was raised. Each stack frame also holds references to the local and global variables in effect at the time each function in the stack was called.

Since there is no way for pickle to know what to serialize and what to ignore, if you were able to pickle a traceback you'd end up pickling a moving snapshot of the entire application state: as pickle runs, other threads may be modifying the values of shared variables.

One solution is to create a picklable object to walk the traceback and extract only the information you need to save.

Answer 2

You can use tblib

    try:
        1 / 0
    except Exception as e:
         raise Exception("foo") from e
except Exception as e:
    s = pickle.dumps(e)
raise pickle.loads(s)

Answer 3

I guess you are interested in saving the complete call context (traceback + globals + locals of each frame).

That would be very useful to determine a difference of behavior of the same function in two different call contexts, or to build your own advanced tools to process, show or compare those tracebacks.

The problem is that pickl doesn't know how to serialize all type of objects that could be in the locals or globals.

I guess you can build your own object and save it, filtering out all those objects that are not picklabe. This code can serve as basis:

import sys, traceback

def print_exc_plus():
    """
    Print the usual traceback information, followed by a listing of all the
    local variables in each frame.
    """
    tb = sys.exc_info()[2]
    while 1:
        if not tb.tb_next:
            break
        tb = tb.tb_next
    stack = []
    f = tb.tb_frame
    while f:
        stack.append(f)
        f = f.f_back
    stack.reverse()
    traceback.print_exc()
    print "Locals by frame, innermost last"
    for frame in stack:
        print
        print "Frame %s in %s at line %s" % (frame.f_code.co_name,
                                             frame.f_code.co_filename,
                                             frame.f_lineno)
        for key, value in frame.f_locals.items():
            print "\t%20s = " % key,
            #We have to be careful not to cause a new error in our error
            #printer! Calling str() on an unknown object could cause an
            #error we don't want.
            try:                   
                print value
            except:
                print "<ERROR WHILE PRINTING VALUE>"

but instead of printing the objects you can add them to a list with your own pickable representation ( a json or yml format might be better).

Maybe you want to load all this call context in order to reproduce the same situation for your function without run the complicated workflow that generate it. I don't know if this can be done (because of memory references), but in that case you would need to de-serialize it from your format.