Python -- efficiency of caught exceptions

Question

Possible Duplicate:
Python FAQ: “How fast are exceptions?”

I remember reading that Python implements a "Better to seek forgiveness than to ask permission" philosophy with regards to exceptions. According to the author, this meant Python code should use a lot of try - except clauses, rather than trying to determine ahead of time if you were about to do something that would cause an exception.

I just wrote some try - except clauses on my web app in which an exception will be raised most of the time the code is run. So, in this case, raising and catching an exception will be the norm. Is this bad from an efficiency point of view? I also remember someone telling me that catching a raised exception has a large performance overhead.

Is it unnecessarily inefficient to use try - except clauses in which you expect an exception to be raised and caught almost all of the time?

Here's the code -- its using the Django ORM to check for objects that associate users with various third party social providers.

try:
    fb_social_auth = UserSocialAuth.objects.get(user=self, provider='facebook')
    user_dict['facebook_id'] = fb_social_auth.uid
except ObjectDoesNotExist:
    user_dict['facebook_id'] = None

try:
    fs_social_auth = UserSocialAuth.objects.get(user=self, provider='foursquare')
    user_dict['foursquare_id'] = fs_social_auth.uid
except ObjectDoesNotExist:
    user_dict['foursquare_id'] = None

try:
    tw_social_auth = UserSocialAuth.objects.get(user=self, provider='twitter')
    user_dict['twitter_id'] = tw_social_auth.uid
except ObjectDoesNotExist:
    user_dict['twitter_id'] = None

The first one will rarely take the exception, since right now we are enforcing "Sign In With Facebook" as the primary method for new users to join the site. But, Twitter and Foursquare are optional, in case they want to import friends or followers, and I expect most people will not.

I'm open to better ways to code this logic.

Applying "Better to seek forgiveness than to ask permission" everywhere where it is applicable is definitely not the best idea. Consider this: if you know that in 95% of calls to the code inside `try..catch` an exception is raised, you'd better rewrite it. — Zaur Nasibov, Jan 19 '13 at 20:23
One saying that goes around is "exceptions are for exceptional cases". I think that if you're expecting an exception most of the time, then you may be using exceptions in the wrong way. — , Jan 19 '13 at 20:32

unutbu · Accepted Answer · 2013-01-19T20:19:04.250

Whenever you code there is a balancing of concerns: performance, readability, correctness, extendability, maintainability, etc. Unfortunately, it is often not possible to improve code in each of these directions at the same time. What is fast may not be as readable for instance.

One of the reasons why try..except is encouraged in Python is because you often can not anticipate all the ways your code may be used, so rather than checking if a specific condition exists, it is more general to just catch any of a certain class of error that might arise. Thus try..except may make your code more reusable.

However, it is also true that try..except is slow if the except clause is often being reached.

Is there a way to code that block so that an exception is not being raised and use try..except to catch the less frequent condition?

Or if not, for the sake of efficiency, you may choose not to use try..except. There are few hard and fast rules in programming. You have to choose your way based on your balance of concerns.

+1 for There are few hard and fast rules in programming. :) – Fredrick Brennan Jan 19 '13 at 20:21 — Fredrick Brennan, Jan 19 '13 at 20:21

score 2 · Answer 2 · answered Jan 19 '13 at 21:06

If you are attempting to optimize this function for speed, you should focus on what is likely to be the actual bottleneck. Your three database queries, each of which will cause the operating system to context switch, almost certainly take an order of magnitude longer than catching an exception. If you want to make the code as fast as possible, begin by combining all three database queries into one:

auth_objects = UserSocialAuth.objects.filter(user=self, provider__in=('facebook', 'foursquare', 'twitter'))

and then loop through the objects. The provider__in filter may be unnecessary if those three providers are the only ones in the database.

Gareth Rees · Answer 3 · 2013-01-19T22:10:55.853

It's true that catching an exception is moderately expensive (see below for some timings) and you wouldn't want to this it in the bottleneck of your program, but in the examples you give, catching the exception is going to be a very small part of the runtime in comparison with the call to Model.objects.get which has to build a SQL query, transmit it to the database server, and wait for the database to report that there's no such object.

Some example timings. Function f2 throws and catches an exception, while f1 implements the same functionality without using exceptions.

d = dict()

def f1():
    if 0 in d: return d[0]
    else: return None

def f2():
    try: return d[0]
    except KeyError: return None

>>> timeit(f1)
0.25134801864624023
>>> timeit(f2)
2.4589600563049316

And f3 tries to get a non-existent object from the database (which is running on the same machine) via Django's ORM:

def f3():
    try:
        MyModel.objects.get(id=999999)
    except MyModel.DoesNotExist:
        pass

This takes about 400 times longer than f2 (so long that I didn't want to wait for the default number=1000000 iterations to complete):

>>> timeit(f3, number=1000)
1.0703678131103516

Python -- efficiency of caught exceptions

3 Answers3

Linked