Python: catch any exception and put it in a variable

Question

To figure out what it would take to avoid some recursion, I need to catch any exception (edit: Not just ones derived from Exception, but all exceptions, including KeyboardInterrupt and user exceptions), put it in a variable, and later re-raise it outside the catch block. Essentially, I'm trying to roll my own finally block. Is this possible?

The actual problem is to call a number of cleanup functions, and if any of them fail, all the others should also be called, then the exception for the one that failed should still propagate. Here's my current solution, it takes a list of Popen objects:

def cleanupProcs(procs):
    if not procs:
        return

    proc = procs.pop(0)
    try:
        proc.terminate()
        proc.wait()
    finally:
        cleanupProcs(procs)

Is there an iterative way to do this? A more elegant way? A more Pythonic way?

What happens if more than one `proc` fails? Which exception do you want to propagate? — , Oct 16 '13 at 14:22
what should happen if more than one fails? Is there a reason not to use a exception hierarchy or a wrapper? — loopbackbee, Oct 16 '13 at 14:28
FWIW, python is not good with recursion, it only has a max recursion depth of 99. I think. — Games Brainiac, Oct 16 '13 at 14:39
@GamesBrainiac The default recursion depth is 1000, and can be increased easily with `sys.setrecursiondepth`. — Henry Keiter, Oct 16 '13 at 17:56
@HenryKeiter just because you _can_ increase the stack size doesn't mean you _should_. There are a number of other good reasons not to use recursion. — Benjamin Hodgson, Oct 16 '13 at 22:06
You might actually try [my plan B question](https://stackoverflow.com/q/18188563/321973) — Tobias Kienzler, Dec 23 '17 at 08:06

score 13 · Accepted Answer · edited May 23 '17 at 12:33

13

If you want the stacktrace included:

try:
    # something
except:
    the_type, the_value, the_traceback = sys.exc_info()

later

raise the_type, the_value, the_traceback

(Related to this answer)

See also here for Python 2.7.

edited May 23 '17 at 12:33

Community

1
1

answered Oct 16 '13 at 14:29

Tobias Kienzler

25,759
22
127
221

Just a note to other readers: this doesn't work for Python 3.5.2. – Roy Dec 23 '17 at 06:10
@Roy I think I only tested this in 2.7, but you might find [my plan B question](https://stackoverflow.com/q/18188563/321973) more helpful, especially [my version-agnostic answer](https://stackoverflow.com/a/18189174/321973) – Tobias Kienzler Dec 23 '17 at 08:07

score 4 · Answer 2 · answered Oct 16 '13 at 14:40

4

In this case, I think I would argue that you may not be doing it right.

To me, the point of an exception is to signal a bum bum baaaaa exceptional circumstance. And when you are writing code that may fire an exception you have two responsible options - catch it and do something with it (like recover), or completely ignore it.

From what you say in your post, you don't really care that an exception happened. It shouldn't halt your program, and program flow should continue as normal. You do however want to know that an exception happened. And this is where the logging module comes in:

import logging
log = logging

def get_some_cheese():
    raise ValueError("Sorry, we're right out.")

try:
    get_some_cheese()
except:
    log.exception("What a waste of life")

When you log an exception it automagically adds stack trace information for you. After you config your logging a bit, you can get it setup to do all sorts of whatever things you want to do - send an email, write to a file or stdout/err, whatever. But then you'll get informed that an exception occured, but you can also simply recover from the error and continue on your merry way cleaning up whatever it is you need to clean up.

answered Oct 16 '13 at 14:40

Wayne Werner

49,299
29
200
290

I want normal processing to stop, I just want all these cleanups to be attempted. – Martin C. Martin Oct 16 '13 at 14:58
Then I think what you're looking for is [atexit](http://docs.python.org/2/library/atexit.html) – Wayne Werner Oct 16 '13 at 15:01
Your point about exceptions being for 'exceptional' circumstances is less true for Python than it is for other languages. For example, the way `for` loops are implemented involves repeatedly calling `next()` on an iterable and catching `StopIteration` when it's time to stop looping. In other words, _every for loop involves throwing an catching an exception_. For a more visible example, see the fact that `dict[key]` throws a `KeyError` for the quite unremarkable case that `key` is missing from the dict (rather than returning `None`). In short, exceptions are not so exceptional in Python! – Benjamin Hodgson Oct 16 '13 at 18:46
@poorsod, I agree that they're certainly less exceptional (and in some cases, especially StopIteration, it's even faster [so I've read] to use exceptions) in Python... but I would still argue that it's better practice to treat exceptions as "exceptional", e.g. use `val = my_dict.get('answer', 42)` rather than `try: val = my_dict['answer'] except KeyError: val = 42`. EAFP doesn't mean that you should explicitly engineer your code so that you're *forced* to ask forgiveness all the time. Though you certainly could - my argument here is that it simply makes for less grokkable code. – Wayne Werner Oct 16 '13 at 19:09
2

I think we're basically on the same page. I do feel that `get()`, being the less common form, does actually have implications to the reader beyond "I don't want this to throw an exception". It suggests that you expect the key to be missing more often than it's there. Catching `KeyError`, on the other hand, expresses the (more common, in my experience) case when you expect the key to be there but you can recover if it's not. (Indeed, catching `KeyError` is more efficient than `get()` if it doesn't get thrown often.) – Benjamin Hodgson Oct 16 '13 at 21:59

Henry Keiter · Answer 3 · 2013-10-16T18:12:42.330

I'd probably use BaseException to catch anything that gets thrown, and iterate through all your cleanup functions (instead of using recursion). Then append any exceptions to a list, to deal with (re-raise, log, etc) as appropriate when you finish the cleanup.

def runCleanup(procs):
    exceptions = []
    for proc in procs:
        try:
            proc.terminate()
            proc.wait()
        except BaseException as e:
            exceptions.append(e) # Use sys.exc_info() for more detail

    return exceptions # To be handled or re-raised as needed

score 1 · Answer 4 · answered Oct 16 '13 at 14:33

1

you can use:

procexceptions = []

except Exception, e:
    procexceptions.append(e)

and then later (after the loop for terminating processes) you can

raise procexceptions[0]

etc.

answered Oct 16 '13 at 14:33

Corley Brigman

11,633
5
33
40

This only catches exceptions derived from Exception. So if someone kills the script using Ctrl-C, your version won't attempt any of the later cleanups , because KeyboardInterrupt isn't derived from Exception. User exceptions needn't be derived from Exception either. – Martin C. Martin Oct 16 '13 at 14:58
2

well, you could use BaseException instead, if that's what you wanted. KeyboardInterrupt _is_ derived from BaseException. Usually you don't, but it sounds like in your case you do. – Corley Brigman Oct 16 '13 at 15:21

score 0 · Answer 5 · answered Oct 16 '13 at 14:27

0

Openstack does something very similar for a different reason. Have a look at https://github.com/openstack/nova/blob/master/nova/openstack/common/excutils.py#L30 (function save_and_reraise_exception). In their case it works like a resource manager.

answered Oct 16 '13 at 14:27

viraptor

33,322
10
107
191

The linked code only catches exceptions derived from Exception. So if someone kills the script using Ctrl-C, your version won't attempt any of the later cleanups , because KeyboardInterrupt isn't derived from Exception. User exceptions needn't be derived from Exception either – Martin C. Martin Oct 16 '13 at 15:01
1

@MartinC.Martin This code doesn't care what you catch. It's the part that you write that decides about the type of exception. Just use `except:` instead of `except Exception:`. It's not a copy-paste solution for your problem. Give it a bit of thought before downvoting everyone not giving you exactly the solution you want. – viraptor Oct 16 '13 at 15:36

score 0 · Answer 6 · answered Oct 16 '13 at 14:29

0

Like just about everything else in Python, exceptions are objects and therefore can be bound to names and operated upon. Here's a short example showing how to grab an exception and use it later:

>>> def to_int(x):
...     try:
...         return int(x)
...     except Exception, e:
...         print 'in exception block:', e
...     print 'after exception block:', e

>>> to_int('12')
12
>>> to_int('abc')
in exception block: invalid literal for int() with base 10: 'abc'
after exception block: invalid literal for int() with base 10: 'abc'

answered Oct 16 '13 at 14:29

Chris Johnson

20,650
6
81
80

This doesn't catch any exception, it only catches ones derived from Exception. So if someone kills the script using Ctrl-C, your version won't attempt any of the later cleanups , because KeyboardInterrupt isn't derived from Exception. User exceptions needn't be derived from Exception either. That's why my question asked about catching *all* exceptions. And a minor point: the "Exception, e" syntax was never official and is discouraged, "Exception as e" is preferred. – Martin C. Martin Oct 16 '13 at 14:54
@MartinC.Martin Just catch `BaseException` instead. – Benjamin Hodgson Oct 16 '13 at 18:49

JadedTuna · Answer 7 · 2013-10-16T17:53:16.227

0

Thats easy:

>>> try:
...     #something
... except BaseException, e: # OK. Using BaseException instead of Exception
...     pass
... 
>>> 
>>> raise e
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>>

edited Oct 16 '13 at 17:53

answered Oct 16 '13 at 14:37

JadedTuna

1,783
2
18
32

1

@MartinC.Martin, Use BaseException then! http://docs.python.org/2/library/exceptions.html#exception-hierarchy – JadedTuna Oct 16 '13 at 17:52

Python: catch any exception and put it in a variable

7 Answers7

Linked