Flask/Nginx/Uwsgi Gateway 504 error, "SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request"

Question

I'm making an app that lets the user upload a csv of data, and generates a plot of that data, saves the plot to a pdf, then returns the plot as a download to the user. The crux of the issue is that in reality, this plot takes 1 - 2 minutes to generate.

The App

The route is as follows:

@app.route("/gen_pdf", methods=["GET","POST"])
def transform2():
    f = request.files['input_file1']
    if not f:
        return "Main input file not found!"

    # Read uploaded data as a Stream into a dataframe. 
    stream = io.StringIO(f.stream.read().decode("UTF8"), newline=None)
    csv_input = csv.reader(stream)
    contents = []
    for row in csv_input:
        contents.append(float(row[0]))

    #Artificially increase time required to generate plot
    time.sleep(60)
    plt.hist(contents, bins=100)
    fn = "test_histogram.pdf"
    plt.savefig(app.config["CLIENT_DIR"]+"/"+fn)

    return send_from_directory(app.config["CLIENT_DIR"], filename=fn, as_attachment=True)

gen_pdf.html, for completeness (this can just be embedded within some html boilerplate)

{% extends "layout.html" %}
{% block content %}
    <h1>Test</h1>
    <hr/>
    <form action="/transform2" method="post" enctype="multipart/form-data">
      <h3>Upload input file (csv)</h3>
      <input type="file" name="input_file1" id="input_file1">
      <hr/>
      <input type="submit" value="Create and Serve PDF" name="submit">
    </form>
{% endblock content %}

The data to be uploaded are just a single csv file with a single column of randomly generated numbers.

Nginx/Uwsgi Setup

These were set up on a Ubuntu 18.04 VM, following the setup here.

The Error

As mentioned, this leads to a gateway 504 error. However, if I reduce the delay in the time.sleep() command to, say, 10s, everything works just fine, so the error somehow stems from something timing out before the pdf plot is completed.

When checking journalctl --unit=<my_project>.service -n 100 --no-pager, I get:

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /transform2 (ip <ip_address>) !!!
uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /transform2 (<ip_address>)
OSError: write error
[pid: 27863|app: 0|req: 156/335] <ip_address> () {50 vars in 1083 bytes} [Mon Jul 27 02:18:38 2020] POST /transform2 => generated 0 bytes in 60835 msecs (HTTP

I'm aware of the explanation of the SIGPIPE error here, but that explanation doesn't seem relevant (it's an internal process, not between visitors).

I've also tried the solutions here and here because they look like similar errors, to no avail.

score 1 · Answer 1 · answered Jul 27 '20 at 06:04

1

For posterity, the solution here was indeed correct: add the line uwsgi_read_timeout 600s; in the appropriate location block.

After that, restart nginx for the new config to take effect: systemctl restart nginx. This was the missing step.

answered Jul 27 '20 at 06:04

AndreyIto

954
1
14
35

I don't have nginx, is there a way to force this using just uwsgi? – DJ_Stuffy_K May 05 '21 at 18:43
@DJ_Stuffy_K yes, you want all three of the uWSGI settings `ignore-sigpipe`, `ignore-write-errors`, and `disable-write-exception`. Each corresponds to inhibiting one of the error lines above. You can typically replicate the issue by hammering on the F5 (refresh) key rapidly on a uWSGI hosted URL. – Ben Sturmfels Sep 21 '21 at 07:08

Flask/Nginx/Uwsgi Gateway 504 error, "SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request"

The App

Nginx/Uwsgi Setup

The Error

1 Answers1

Linked