63

I'm running django on gunicorn+nginx. I'm facing a problem with file uploads. Actually uploads are working fine but gunicorn times out thus causing this in nginx:

2011/07/25 12:13:47 [error] 15169#0: *2317 upstream timed out (110: Connection timed out) while reading response header from upstream, client: IP-ADDRESS, server: SERVER, request: "GET /photos/events/event/25 HTTP/1.1", upstream: "http://127.0.0.1:29000/photos/events/event/25", host: "HOST", referrer: "REFERER_ADDRESS"

If I refresh the page, I can see all the photos are uploaded just fine. The problem is that it causes a timeout thus giving the impression that upload did not work.

here is my gunicorn conf:

bind = "127.0.0.1:29000"
logfile = "/path/to/logs/gunicorn.log"
workers = 3

I tried changing timeout but it didn't work.

ivanleoncz
  • 9,070
  • 7
  • 57
  • 49
  • How long does it take for these files to upload before you get the error? – Ken Cochrane Jul 25 '11 at 14:48
  • anywhere between a few seconds to a minute depending on the size of file and my connection speed. (my speed varies a lot :) ). I even get these for 20kb files sometimes. –  Jul 25 '11 at 17:11
  • Is there some sort of socket or communication primitive that you're forgetting to close? – sholsapp Feb 28 '12 at 18:47

5 Answers5

94

You could try upgrading the timeout for your proxy pass in Nginx by adding:

proxy_connect_timeout 75s;
proxy_read_timeout 300s;

on /var/nginx/sites-available/[site-config] or /var/nginx/nginx.conf if you want to increase the timeout limite on all sites served by nginx.

You must add --timeout 300 as well to your gunicorn process/config.

This solved my problems in the past with bigger uploads.

kontextify
  • 478
  • 5
  • 16
fijter
  • 17,607
  • 2
  • 25
  • 28
  • 7
    proxy_connect_timeout <= 75s. http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout – zubinmehta Jan 25 '13 at 19:53
  • 3
    i would plus one this several times. This is the reason I had the whole headache will all file uploads and spent hours double checking my code and breaking my settings.py several times. I am moving back to uwsgi instead. – Abhishek Dujari Apr 27 '13 at 19:48
  • 2
    For the record, I was having some large-file/processing time issues, and adjusting the "timeout" value in the gunicorn config is exactly what I needed to solve it. Thanks for mentioning that! – anonymous coward Sep 27 '13 at 17:28
  • 5
    Only need to adjust `proxy_read_timeout 300s;` on nginx config and `-t 300` on gUnicorn command line. Works without increasing the `proxy_connect_timeout` parameter and it is not recommended according to nginx doc (as seen in zm1 comment above). – laurent Oct 29 '14 at 21:38
46

This is not an nginx timeout, but probably a Gunicorn timeout. Gunicorn defaults to a 30 second timeout.

In general, you should fix this by not having an endpoint that takes longer than 30 seconds to return, but if it is a seldom used endpoint, you can also just increase the gunicorn timeout.

If you do this, you should probably also increase the number of gunicorn worker processes as well.

To increase the timeout and workers for gunicorn, you can add the following command-line options on start:

gunicorn --timeout 120 --workers <NUMBER OF WORKER YOU WANT>
Chalist
  • 3,160
  • 5
  • 39
  • 68
ddipasquo
  • 545
  • 4
  • 7
  • 7
    True but `proxy_read_timeout` on nginx has a default of 60s so if your process in gUnicorn takes more than that to reply, it needs to be increased also. http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout – laurent Oct 29 '14 at 21:48
16

We had the same problem using Django+nginx+gunicorn. From Gunicorn documentation we have configured the graceful-timeout that made almost no difference.

After some testings, we found the solution, the parameter to configure is: timeout (And not graceful timeout). It works like a clock..

So, Do:

1) open the gunicorn configuration file

2) set the TIMEOUT to what ever you need - the value is in seconds

NUM_WORKERS=3
TIMEOUT=120

exec gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--timeout $TIMEOUT \
--log-level=debug \
--bind=127.0.0.1:9000 \
--pid=$PIDFILE
Amit Talmor
  • 7,174
  • 4
  • 25
  • 29
0

If you are using reverse proxy with Nginx and running the Django app using Gunicorn, try to adjust Gunicorn timeout parameter and proxy_read_timeout parameter from default values.

Fathima Shaibal
  • 15
  • 4
-2

This might help someone with similar problem.

I was getting timeout error from nginx and gunicorn on my Django application. Since I was getting the timeout error from nginx, I couldn't see the real error from Django. After adding the new timeout like fijter suggested. I could see that the error was in the settings.py file.

If you set the DEBUG to False, and didn't add the domain name in the ALLOWED_HOSTS you might get this error.

I just added the domain in the ALLOWED_HOSTS in settings.py and the error was gone.

Very simple solution!

RLott
  • 330
  • 4
  • 12