49

You may know this recommendation from Microsoft about the use of exceptions in .NET:

Performance Considerations

...

Throw exceptions only for extraordinary conditions, ...

In addition, do not throw an exception when a return code is sufficient...

(See the whole text at http://msdn.microsoft.com/en-us/library/system.exception.aspx.)

As a point of comparison, would you recommend the same for Python code?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
luc
  • 41,928
  • 25
  • 127
  • 172
  • 9
    -1: Why are you trying to apply .Net framework library documentation to Python? What made you think this .Net advice applied to Python? Was there some note or hint that this was relevant to Python? – S.Lott Jul 20 '09 at 09:54
  • 44
    I think in fairness he is asking whether "do not throw an exception when a return code is sufficient" applies more generally outside the .Net framework too. It's still a reasonable question even if the answer was "no, it's a .Net thing" – Draemon Jul 20 '09 at 10:17
  • 11
    This is a very general recommendation which applies for .NET and c++. I just wanted if such a recommendation is also valid for python. I know that Python and .NET are different. but both of them have exception handling. It is a point of comparison – luc Jul 20 '09 at 10:18
  • 1
    @luc: It may be a point of comparison, but the .NET quote doesn't apply to Python at all. Perhaps the question should be fixed to clarify this. – S.Lott Jul 20 '09 at 14:51

6 Answers6

43

The pythonic thing to do is to raise and handle exceptions. The excellent book "Python in a nutshell" discusses this in 'Error-Checking Strategies' in Chapter 6.

The book discusses EAFP ("it's easier to ask forgiveness than permission") vs. LBYL ("look before you leap").

So to answer your question:

No, I would not recommend the same for python code. I suggest you read chapter 6 of Python in a nutshell.

codeape
  • 97,830
  • 24
  • 159
  • 188
  • 3
    "in most cases"? Almost nothing returns status codes. If there are any status code returns, it is because of an underlying C-language thing that wasn't properly wrapped in an exception. I'd vote for dropping "in most cases." – S.Lott Jul 20 '09 at 09:52
  • 8
    This answer doesn't explain anything. Can it be expanded to exclude requirement to buy the book? – anatoly techtonik Jun 23 '13 at 09:23
16

The best way to understand exceptions is "if your method can't do what its name says it does, throw." My personal opinion is that this advice should be applied equally to both .NET and Python.

The key difference is where you have methods that frequently can't do what their name says they should do, for instance, parsing strings as integers or retrieving a record from a database. The C# style is to avoid an exception being thrown in the first place:

int i;
if (Int32.TryParse(myString, out i)) {
    doWhatever(i);
}
else {
    doWhatever(0);
}

whereas Python is much more at ease with this kind of thing:

try:
    i = int(myString)
except ValueError:
    i = 0
doWhatever(i);
jammycakes
  • 5,780
  • 2
  • 40
  • 49
10

Usually, Python is geared towards expressiveness.
I would apply the same principle here: usually, you expect a function to return a result (in line with its name!) and not an error code.
For this reason, it is usually better raising an exception than returning an error code.

However, what is stated in the MSDN article applies to Python as well, and it's not really connected to returning an error code instead of an exception.
In many cases, you can see exception handling used for normal flow control, and for handling expected situations. In certain environments, this has a huge impact on performance; in all environments it has a big impact on program expressiveness and maintainability.

Exceptions are for exceptional situations, that are outside of normal program flow; if you expect something will happen, then you should handle directly, and then raise anything that you cannot expect / handle.

Of course, this is not a recipe, but only an heuristic; the final decision is always up to the developer and onto the context and cannot be stated in a fixed set of guidelines - and this is much truer for exception handling.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
rob
  • 36,896
  • 2
  • 55
  • 65
8

In Python exceptions are not very expensive like they are in some other languages, so I wouldn't recommend trying to avoid exceptions. But if you do throw an exception you would usually want catch it somewhere in your code, the exception being if a fatal error occurs.

googletorp
  • 33,075
  • 15
  • 67
  • 82
5

I think whether to return an error code or throw an exception is something very valid to think about, and a cross-linguistic comparison may be helpful and informative. I guess the very generalized answer to this concern is simply the consideration: that the set of legal return values for any function should be made as small as possible, and as large as necessary.

Generally, this will mean that if a given method returns an integer number in a single test case, users can rightfully expect the method to always return an integer number or throw an exception. But, of course, the conceptually simplest way is not always the best way to handle things.

The return-value-of-least-surprise is usually None; and if you look into it, you’ll see that it’s the very semantics of None that license its usage across the board: it is a singleton, immutable value that, in a lot of cases, evaluates to False or prohibits further computation—no concatenation, no arithmetics. So if you chose to write a frob(x) method that returns a number for a string input, and None for non-numeric strings and any other input, and you use that inside an expression like a=42+frob('foo'), you still get an exception very close to the point where bogus things happened. Of course, if you stuff frob('foo') into a database column that has not been defined with NOT NULL, you might run into problems perhaps months later. This may or may not be justifiable.

So in most cases where you e.g. want to derive a number from a string, using somwething like a bare float(x) or int(x) is the way to go, as these built-ins will raise an exception when not given a digestable input. If that doesn’t suit your use case, consider returning None from a custom method; basically, this return value tells consumers that ‘Sorry, I was unable to understand your input.’. But you only want to do this if you positively know that going on in your program does make sense from that point onwards.

You see, I just found out how to turn each notice, warning, and error message into a potentially show-stopping exception in, uhm, PHP. It just drives me crazy that a typo in a variable name generates in the standard PHP configuration, nothing but a notice to the user. This is so bad. The program just goes on doing things with a program code that does not make sense at all! I can’t believe people find this a feature.

Likewise, one should view it like this: if, at any given point in time, it can be asserted with reasonable costs that the execution of a piece of code does no more make sense — since values are missing, are out of bounds, or are of an unexpected type, or when resources like a database connection have gone down — it is imperative, to minimize debugging headaches, to break execution and hand control up to any level in the code which feels entitled to handle the mishap.

Experience shows that refraining from early action and allowing bogus values to creep into your data is good for nothing but making your code harder to debug. So are many examples of over-zealous type-casting: allowing integers to be added to floats is reasonable. To allow a string with nothing but digits to be added to a number is a bogus practice that is likely to create strange, unlocalized errors that may pop up on any given line that happens to be processing that data.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
flow
  • 116
  • 2
2

I did a simple experiment to compare the performance of raising exceptions with the following code:

from functools import wraps
from time import time
import logging

def timed(foo):
    @wraps(foo)
    def bar(*a, **kw):
        s = time()
        foo(*a, **kw)
        e = time()
        print '%f sec' % (e - s)
    return bar

class SomeException(Exception):
    pass

def somefunc(_raise=False):
    if _raise:
        raise SomeException()
    else:
        return

@timed
def test1(_reps):
    for i in xrange(_reps):
        try:
            somefunc(True)
        except SomeException:
            pass

@timed
def test2(_reps):
    for i in xrange(_reps):
        somefunc(False)

def main():

    test1(1000000)
    test2(1000000)

    pass

if __name__ == '__main__':
    main()

With the following results:

  • Raising exceptions: 3.142000 sec
  • Using return: 0.383000 sec

Exceptions are about 8 times slower than using return.

chnrxn
  • 1,349
  • 1
  • 16
  • 17
  • 1
    I ran your test and found about a 4.4x factor, which is not different enough from your results to change any conclusions. But I would state this more precisely as raising and catching an exception is about 2.8 microseconds slower than returning a value and checking it, using your results. On a more modern system with faster CPU that I ran the test on, the difference was 0.34 microseconds. Point being, as you provide evidence for, there is a cost to raising and catching exceptions for large iteration counts, which is one aspect of deciding an exception strategy. – mklein9 Mar 15 '16 at 21:42
  • Sorry for the second comment, but I ran out of time to edit the previous one. My results above are for Python 3.4; 2.7 shows a loop time difference for the exception path of 0.48 microseconds, more than 40% higher than 3.4's, and a ratio of 5.7x. – mklein9 Mar 15 '16 at 21:59
  • 2
    And how much slower, not to mention less readable, is the client code because it has to actually test the result before using it? – holdenweb Aug 02 '16 at 13:05
  • 1
    One consideration is that one generally does not throw exceptions in a loop. This answer says that throwing is more expensive than returning, but I believe it misses the point that you rarely throw, and you do it when there is a problem. The successful path is not really affected by that. If you find yourself in a situation where you throw in a loop, then that's likely a design issue, indeed. – JonasVautherin Jan 07 '19 at 10:16
  • this is a faulty test — the exception path always constructs a new object, but the return path does not. if i `return SomeException()` instead, the difference in speed is cut in half on my machine. – Eevee Mar 13 '21 at 21:54
  • Of course, one does not generally throw exceptions in a loop. Running many iterations is part of the experiment, because we are expected to take the average of multiple readings instead of just taking one measurement. Divide the timings by 1000000 if you will, and the ratio of the two numbers will still be the same. – chnrxn Jun 17 '21 at 15:25