1

I'm now familiar with the general cause of this problem from another SO answer and from the uWSGI documentation, which states:

If an HTTP request has a body (like a POST request generated by a form), you have to read (consume) it in your application. If you do not do this, the communication socket with your webserver may be clobbered.

However, I don't understand what exactly is happening at the TCP level for this problem to occur. Not knowing the details of this process, I would assume the server can simply discard what remains in the stream, but that's obviously not the case.

If I consume only part of the request body in my application and ultimately return a 200 response, a web browser will report a connection reset error. Who reset the connection? The webserver or the client? It seems like all the data has been sent by the client already, but the application has just not exhausted the stream. Is there something that happens when the stream is exhausted in the application that triggers the webserver to indicate it has finished reading?

My application is Python/Flask, but I've seen questions about this from several languages and frameworks. For example, this fails if exhaust() is not called on the request stream:

@app.route('/upload', methods=['POST'])
def handle-upload():
    file = request.stream
    pandas.read_csv(file, nrows=100)
    response = # Do stuff
    file.exhaust()
    return jsonify(response)
wst
  • 11,681
  • 1
  • 24
  • 39
  • Have you checked request.files? – fiacre Oct 30 '19 at 20:17
  • @fiacre This is a raw upload, not a `multipart/form-encoded` request. The multipart decoder in werkzeug is too slow to handle anything but small files. – wst Oct 30 '19 at 20:26

1 Answers1

1

While there is some buffering throughout the chain, large file transfers are not going to complete until the receiver has consumed them. The buffers will fill up, and packets will be dropped until the buffers are drained. Eventually, the browser will give up trying to send the file and drop the connection.

Aaron Bentley
  • 1,332
  • 8
  • 14