Catching Python runtime errors only

Question

I find myself handling exceptions without specifying an exception type when I call code that interacts with system libraries like shutil, http, etc, all of which can throw if the system is in an unexpected state (e.g. a file locked, network unavailable, etc)

try:
    # call something
except:
    print("OK, so it went wrong.")

This bothers me because it also catches SyntaxError and other exceptions based in programmer error, and I've seen recommendations to avoid such open-ended exception handlers.

Is there a convention that all runtime errors derive from some common exception base class that I can use here? Or anything that doesn't involve syntax errors, module import failures, etc.? Even KeyError I would consider a bug, because I tend to use dict.get() if I'm not 100% sure the key will be there.

I'd hate to have to list every single conceivable exception type, especially since I'm calling a lot of supporting code I have no control over.

UPDATE: OK, the answers made me realize I'm asking the wrong question -- what I'm really wondering is if there's a Python convention or explicit recommendation for library writers to use specific base classes for their exceptions, so as to separate them from the more mundane SyntaxError & friends.

Because if there's a convention for library writers, I, as a library consumer, can make general assumptions about what might be thrown, even if specific cases may vary. Not sure if that makes more sense?

UPDATE AGAIN: Sven's answer finally led me to understand that instead of giving up and catching everything at the top level, I can handle and refine exceptions at the lower levels, so the top level only needs to worry about the specific exception type from the level below.

Thanks!

Answer 1

1. Always make the try block as small as possible.

2. Only catch the exceptions you want to handle. Look in the documentation of the functions you are dealing with.

This ensures that you think about what exceptions may occur, and you think about what to do if they occur. If something happens you never thought about, chances are your exception handling code won't be able to correctly deal with that case anyway, so it would be better the exception gets propagated.

You said you'd "hate to have to list every single conceivable exception type", but usually it's not that bad. Opening a file? Catch IOError. Dealing with some library code? They often have their own exception hierarchies with a specific top-level exception -- just catch this one if you want to catch any of the library-specific exceptions. Be as specific as possible, otherwise errors will pass unnoticed sooner or later.

As for convention about user-defined exceptions in Python: They usually should be derived from Exception. This is also what most user-defined exceptions in the standard library derive from, so the least you should do is use

except Exception:

instead of a bare except clause, which also catches KeyboardInterrupt and SystemExit. As you have noted yourself, this would still catch a lot of exceptions you don't want to catch.

Answer 2

Check out this list of built-in exceptions in the Python docs. It sounds like what you mostly want is to catch StandardError although this also includes KeyError. There are also other base classes, such as ArithmeticError and EnvironmentError, that you may find useful sometimes.

I find third-party libraries do derive their custom exceptions from Exception as documented here. Programmers also tend to raise standard Python exceptions such as TypeError, ValueError, etc. in appropriate situations. Unfortunately, this makes it difficult to consistently catch library errors separately from other errors derived from Exception. I wish Python defined e.g. a UserException base class. Some libraries do declare a base class for their exceptions, but you'd have to know what these are, import the module, and catch them explicitly.

Of course, if you want to catch everything except KeyError and, say, IndexError, along with the script-stopper exceptions, you could do this:

try:
    doitnow()
except (StopIteration, GeneratorExit, KeyboardInterrupt, SystemExit):
    raise    # these stop the script
except (KeyError, IndexError):
    raise    # we don't want to handle these ones
except Exception as e:
    handleError(e)

Admittedly, this becomes a hassle to write each time.

Answer 3

How about RuntimeError: http://docs.python.org/library/exceptions.html#exceptions.RuntimeError

If that isn't what you want (and it may well not be), look at the list of exceptions on that page. If you're confused by how the hierarchy fits together, I suggest you spend ten minutes examining the __bases__ property of the exceptions you're interested in, to see what base classes they share. (Note that __bases__ isn't closed over the whole hierarchy - you may need to examine superclass bases also).