RESTful api design: handle exceptions through nested functions (python, flask)

Question

I would like to improve my coding style with a more robust grasp of try, except and raise in designing API, and less verbose code.

I have nested functions, and when one catches an execption, I am passing the exception to the other one and so on.

But like this, I could propagate multiple checks of a same error. I am referring to: [Using try vs if in python for considering cost of try operation.

How would you handle an error only once across nested functions ?

E.g.

• I have a function f(key) doing some operations on key; result is passed to other functions g(), h()
• if result comply with expected data structure, g() .. h() will manipulate and return updated result
• a decorator will return final result or return the first error that was met, that is pointing out in which method it was raised (f(),g() or h()).

I am doing something like this:

def f(key):
   try:
     #do something
     return {'data' : 'data_structure'}
   except:
     return {'error': 'there is an error'}

@application.route('/')
def api_f(key):
    data = f(k)
    try:
       # do something on data
       return jsonify(data)
    except:
       return jsonify({'error':'error in key'})

Answer 1

IMO try/except is the best way to go for this use case. Whenever you want to handle an exceptional case, put in a try/except. If you can’t (or don’t want to) handle the exception in some sane way, let it bubble up to be handled further up the stack. Of course there are various reasons to take different approaches (e.g. you don’t really care about an error and can return something else without disrupting normal operation; you expect “exceptional” cases to happen more often than not; etc.), but here try/except seems to make the most sense:

In your example, it’d be best to leave the try/except out of f() unless you want to…

raise a different error (be careful with this, as this will reset your stack trace):

try:
    ### Do some stuff
except:
    raise CustomError('Bad things')

do some error handling (e.g. logging; cleanup; etc.):

try:
    ### Do some stuff
except:
    logger.exception('Bad things')
    cleanup()

    ### Re-raise the same error
    raise

Otherwise, just let the error bubble up.

Subsequent functions (e.g. g(); h()) would operate the same way. In your case, you’d probably want to have some jsonify helper function that jsonifies when possible but also handles non-json data:

def handle_json(data):
    try:
        return json.dumps(data)
    except TypeError, e:
        logger.exception('Could not decode json from %s: %s', data, e)

        # Could also re-raise the same error
        raise CustomJSONError('Bad things')

Then, you would have handler(s) further up the stack to handle either the original error or the custom error, ending with a global handler that can handle any error. In my Flask application, I created custom error classes that my global handler is able to parse and do something with. Of course, the global handler is configured to handle unexpected errors as well.

For instance, I might have a base class for all http errors…

### Not to be raised directly; raise sub-class instances instead
class BaseHTTPError(Exception):
    def __init__(self, message=None, payload=None):
        Exception.__init__(self)
        if message is not None:
            self.message = message
        else:
            self.message = self.default_message

        self.payload = payload

    def to_dict(self):
        """ 
        Call this in the the error handler to serialize the
        error for the json-encoded http response body.
        """
        payload = dict(self.payload or ()) 
        payload['message'] = self.message
        payload['code'] = self.code
        return payload

…which is extended for various http errors:

class NotFoundError(BaseHTTPError):
    code = 404 
    default_message = 'Resource not found'

class BadRequestError(BaseHTTPError):
    code = 400 
    default_message = 'Bad Request'

class NotFoundError(BaseHTTPError):
    code = 500 
    default_message = 'Internal Server Error'

### Whatever other http errors you want

And my global handler looks like this (I am using flask_restful, so this gets defined as a method on my extended flask_restful.Api class):

class RestAPI(flask_restful.Api):
  def handle_error(self, e):
      code = getattr(e, 'code', 500)
      message = getattr(e, 'message', 'Internal Server Error')
      to_dict = getattr(e, 'to_dict', None)

      if code == 500:
          logger.exception(e)

      if to_dict:
          data = to_dict()
      else:
          data = {'code': code, 'message': message}

      return self.make_response(data, code)

With flask_restful, you may also just define your error classes and pass them as a dictionary to the flask_restful.Api constructor, but I prefer the flexibility of defining my own handler that can add payload data dynamically. flask_restful automatically passes any unhandled errors to handle_error. As such, this is the only place I’ve needed to convert the error to json data because that is what flask_restful needs in order to return an https status and payload to the client. Notice that even if the error type is unknown (e.g. to_dict not defined), I can return a sane http status and payload to the client without having had to convert errors lower down the stack.

Again, there are reasons to convert errors to some useful return value at other places in your app, but for the above, try/except works well.