1

I am trying to catch an exception when using a Response object. My use case is that I want to stream data like described in the official documentation and that there is the possibility of an exception when I get the data that I want to stream: 

# test.py
from flask import Flask, Response, jsonify
import werkzeug

app = Flask(__name__)

@app.route("/")
def hello():
    def generate():    
        raise Exception                                                                                                                                                                                                                                           
        for i in range(0,10): 
            yield '1'    

    try:
        return Response(generate())
    except Exception:
        return jsonify("error") # expected this but instead 500 server error is returned

if __name__ == '__main__':
    app.run()

When I then run the server and request the data, I would expect to see "error" but instead an Internal Server Error message from Flask is shown.

FLASK_APP=test.py flask run
curl http://localhost:5000/

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to 
complete your request.  Either the server is overloaded or there is 
an error in the application.</p>

In some way the exception gets raised, as we can see it in stdout but does not get catched by Flask. I also tried to make sure that the exception does not get catched by setting passthrough_errors option of the werkzeug server. Then the exception does not get catched in Werkzeug e.g. here or here. Unfortunately this did not helped for catching it in the Flask application above:

app.run(passthrough_errors=True)

Output of flask --version:

Flask 1.0.2
Python 3.7.0 (default, Aug 22 2018, 15:22:33) 
[Clang 9.1.0 (clang-902.0.39.2)]

UPDATE: This problem also occurs when I am not using the streaming:

from flask import Flask, Response, jsonify

app = Flask(__name__)

@app.route("/")
def hello():
    try:
        return Response(2)
    except Exception as error:
        return jsonify({'error': error})

if __name__ == '__main__':
    app.run(passthrough_errors=False)

A TypeError gets raised on Flask side, because the Integer is not iterable but the exception does not get catched.

FLASK_APP=test.py flask run
curl http://localhost:5000/

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to 
complete your request.  Either the server is overloaded or there is 
an error in the application.</p>
Hannes
  • 31
  • 1
  • 5

2 Answers2

2

Let's change your code and run it again:

@app.route("/")
def hello():
    def generate():    
        raise Exception                                                                                                                                                                                                                                           
        yield '1'    

    try:
        resp = Response(generate())
        data = resp.data
        return resp

    except Exception:
        return jsonify("error") # expected this but instead 500 server error is returned

This code will correctly return "error", the previous one also "correctly" raises a 500 server error. To understand why, you have to think of the execution flow. In your initial code, return Response(generate()) will return immediately without executing the generator function. The consumer, in this case probably werkzeug, will try to read data or a similar property which will then cause the generator to run. By that time your function is already executed and of course no exception has happened, since you are returning the generator wrapped in a Response. Hope this helps.

Update

The above code only demonstrates the problem and explains why the exception is not caught in the original code. If you have an error in the middle of your stream IMHO the application should give a 500 server error and die. Moving the exception to the generator however will let you catch such errors:

@app.route("/")
def hello():
    def generate():
        try:
            raise Exception('some error')                                                                                                                                                                                                                                           
            for i in range(0,10):
                yield '1'
        except Exception as e:
                yield str(e)

    return Response(generate())

And you can't use jsonify because the generator is being consumed outside of your application context.

mehdix
  • 4,984
  • 1
  • 28
  • 36
  • Thanks for the fast answer. This seems to work but I am not sure if it is still streaming because of the assignment? Because actually I am trying to stream a lot of JSON data with this/ – Hannes Nov 26 '18 at 16:31
  • I used the examples from the [official documentation](http://flask.pocoo.org/docs/1.0/patterns/streaming/?highlight=stream). That's why I am wondering – Hannes Nov 26 '18 at 16:55
  • 1
    @Hannes this was only a sample to demonstrate the problem. You can't catch exceptions, there is a reason there is no try/catch in the official docs for streaming example. – mehdix Nov 26 '18 at 22:59
2

In case you come here from a web search because your exceptions in a generator don't get rendered on the client side: That's because the pattern of generators in responses is really HTTP chunked streaming. This technique results in one HTTP response being sent to the client in chunks. The HTTP response code (e.g. 200 for success) is returned first. So if any of the following chunks trigger an exception in your code, Flask has no possibility of returning that to the browser.

Hubert Grzeskowiak
  • 15,137
  • 5
  • 57
  • 74