Python FAQ: “How fast are exceptions?”

Question

I was just looking at the Python FAQ because it was mentioned in another question. Having never really looked at it in detail before, I came across this question: “How fast are exceptions?”:

A try/except block is extremely efficient. Actually catching an exception is expensive. In versions of Python prior to 2.0 it was common to use this idiom:
try:
    value = mydict[key]
except KeyError:
    mydict[key] = getvalue(key)
    value = mydict[key]

I was a little bit surprised about the “catching an exception is expensive” part. Is this referring only to those except cases where you actually save the exception in a variable, or generally all excepts (including the one in the example above)?

I’ve always thought that using such idioms as shown would be very pythonic, especially as in Python “it is Easier to Ask Forgiveness than it is to get Permission”. Also many answers on SO generally follow this idea.

Is the performance for catching Exceptions really that bad? Should one rather follow LBYL (“Look before you leap”) in such cases?

(Note that I’m not directly talking about the example from the FAQ; there are many other examples where you just look out for an exception instead of checking the types before.)

score 22 · Accepted Answer · answered Nov 12 '11 at 23:45

22

Catching exceptions is expensive, but exceptions should be exceptional (read, not happen very often). If exceptions are rare, try/catch is faster than LBYL.

The following example times a dictionary key lookup using exceptions and LBYL when the key exists and when it doesn't exist:

import timeit

s = []

s.append('''\
try:
    x = D['key']
except KeyError:
    x = None
''')

s.append('''\
x = D['key'] if 'key' in D else None
''')

s.append('''\
try:
    x = D['xxx']
except KeyError:
    x = None
''')

s.append('''\
x = D['xxx'] if 'xxx' in D else None
''')

for i,c in enumerate(s,1):
    t = timeit.Timer(c,"D={'key':'value'}")
    print('Run',i,'=',min(t.repeat()))

Output

Run 1 = 0.05600167960596991       # try/catch, key exists
Run 2 = 0.08530091918578364       # LBYL, key exists (slower)
Run 3 = 0.3486251291120652        # try/catch, key doesn't exist (MUCH slower)
Run 4 = 0.050621117060586585      # LBYL, key doesn't exist

When the usual case is no exception, try/catch is "extremely efficient" when compared to LBYL.

answered Nov 12 '11 at 23:45

Mark Tolonen

166,664
26
169
251

Pretty much what I was going to say. It's using exceptions to control your normal program flow that's really expensive. – Tony Hopkinson Nov 12 '11 at 23:53
3

Doing some quick math, the numbers indicate try/except should only be used if you expect the key won't be found at most 8.95% of time (about 1 in 11 calls). I don't have room to derive the equation I used but: let `cbn` = cost of branching method in normal case, `cbx` = cost of branch method in exceptional case, `ctn` = cost of try/except in normal case, `ctx` = cost of try/except in exceptional case, and px = probability of exceptional case occurring; then for all `px <= (ctn-cbn)/(ctn-ctx+cbx-cbn)`, the try/except method will be faster in the long term. – Eli Collins Nov 13 '11 at 03:32
Just an addendum - since measuring and calculating these sorts of things tends to get time-consuming in itself, I try to use a rule of thumb for python that anything which occurs more than 1/8 times is not an "exceptional" case, and should probably use another method - which has also proved useful when designing an api, as well as writing code. – Eli Collins Nov 13 '11 at 03:43
1

"exceptions should be exceptional" is a good mantra. I forget it too often. – kevinarpe Nov 08 '14 at 07:46

score 6 · Answer 2 · answered Nov 12 '11 at 21:37

The cost depends on implementation, obviously, but I wouldn't worry about it. It's unlikely going to matter, anyway. Standard protocols raise exceptions in strangest of places (think StopIteration), so you're surrounded with raising and catching whether you like it or not.

When choosing between LBYL and EAFP, worry about readability of the code, instead of focusing on micro-optimisations. I'd avoid type-checking if possible, as it might reduce the generality of the code.

Yes, avoid premature optimization. – Brecht Machiels Sep 05 '16 at 14:25 — Brecht Machiels, Sep 05 '16 at 14:25

score 0 · Answer 3 · answered Dec 05 '11 at 20:45

0

If the case where the key is not found is more than exceptional, I would suggest using the 'get' method, which provide a constant speed in all cases :

s.append('''\
x = D.get('key', None)
''')

s.append('''\
x = D.get('xxx', None)
''')

answered Dec 05 '11 at 20:45

Johan Boulé

1,936
15
19

You can use `dict.setdefault` instead. But then again, the code was just a simple example from the FAQ. – poke Dec 05 '11 at 21:10

Python FAQ: “How fast are exceptions?”

3 Answers3

Output

Linked

Related