Django Broken pipe in Debug mode

Question

I have django 1.3 on the remote server behind Nginx.

If I run django with apache + mod_wsgi, I can watch errors in apache log files. It's ok but I'd like to have in console.

If I run django own development server, I get errors with stacktrace in console only when DEBUG = False. In DEBUG mode console outputs

Exception happened during processing of request from (..., ...)
Traceback (most recent call last):
  File "/usr/local/python/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/local/python/lib/python2.7/SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "/usr/local/python/lib/python2.7/SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/python/lib/python2.7/site-packages/django/core/servers/basehttp.py", line 570, in __init__
    BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
  File "/usr/local/python/lib/python2.7/SocketServer.py", line 641, in __init__
    self.finish()
  File "/usr/local/python/lib/python2.7/SocketServer.py", line 694, in finish
    self.wfile.flush()
  File "/usr/local/python/lib/python2.7/socket.py", line 301, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe

I want to figure out why? Why does django just output unnamed Exception? Why does it depend on DEBUG variable.

This errors occurs mostly outside views when I have no access to request object. So I can't catch it in middleware or using logging handler.

UPDATE. I noticed if I request to django server directly I never get Broken pipe. So may the issue occur while Nginx proxy django?

I haven't had this issue on a live server, however it occurs from time to time on the django dev server on my local computer.. I haven't been able to resolve this either — Hedde van der Heide, Oct 27 '11 at 07:37
It could just be NGINX timing out before django sends the response. What's the "proxy_read_timeout" set to in your nginx.conf file? — Nils, Jan 02 '14 at 00:47
Not sure if it adds any valuable info, but I've been working on a local project (for practise purposes) for about a month without seeing the "broken pipe" message. As soon as I generated a new SECRET_KEY and added it to `settings.py`, I started getting this message. — Floella, Apr 03 '23 at 14:26

score 78 · Answer 1 · answered Oct 27 '11 at 08:03

This isn't really an issue with your site, more with the Django devserver: see this Django ticket. To put it bluntly, just ignore it as it is a known error, and won't be fixed.

In that ticket's comments a quite clear explanation is given:

According to many sources the 'Broken Pipe' is a normal browser quirk. For example, the browser reads from the socket and then decides that the image it's been reading apparently didn't change. The browser now this (forcefully) closes the connection because it does not need more data. The other end of this socket (the python runserver) now raises a socket exception telling the program that the client 'Broke the socket pipe'.

Thank you. I've seen that old ticket before. I've just updated question. — San4ez, Oct 28 '11 at 06:43

score 12 · Accepted Answer · answered Oct 28 '11 at 08:45

12

Nginx directive proxy_intercept_errors off; (disabled by default) is what I needed

answered Oct 28 '11 at 08:45

San4ez

8,091
4
41
62

Hi San4ez, as you say this appears to be disabled by default. How did explicitly disabling this fix your problem? (I might have a related issue...) – Alex Aug 05 '13 at 08:24
I'm not quite understand your question. Imagine, nginx proxies request to django upstream and gets some error status (400 or higher). If `proxy_intercept_errors` is disabled, nginx responses error page as is from django. Otherwise nginx stops reading django response and send to user custom error page. Django server (or any other) throws error that client doesn't read socket - broken pipe – San4ez Aug 05 '13 at 14:21
Understood; and actually this worked for me. It is just odd because if proxy_intercept_errors is off by default, explicitly turning it off should have no effect. See http://wiki.nginx.org/HttpProxyModule#proxy_intercept_errors – Alex Aug 06 '13 at 06:17
Yes, sure. But I had nginx config where this options was enabled by someone – San4ez Aug 06 '13 at 06:21
Right. That makes sense then. Weirdly, it worked for me too, so I guess that key must be generated in some file somewhere in some builds. – Alex Aug 07 '13 at 00:17

Rick Mohr · Answer 3 · 2014-04-01T14:55:43.637

The nginx directive (checked answer) didn't work for me, but combining monkey patches from Igor Katson and Michael_Scharf did:

def patch_broken_pipe_error():
    """Monkey Patch BaseServer.handle_error to not write
    a stacktrace to stderr on broken pipe.
    http://stackoverflow.com/a/22618740/362702"""
    import sys
    from SocketServer import BaseServer
    from wsgiref import handlers

    handle_error = BaseServer.handle_error
    log_exception = handlers.BaseHandler.log_exception

    def is_broken_pipe_error():
        type, err, tb = sys.exc_info()
        return repr(err) == "error(32, 'Broken pipe')"

    def my_handle_error(self, request, client_address):
        if not is_broken_pipe_error():
            handle_error(self, request, client_address)

    def my_log_exception(self, exc_info):
        if not is_broken_pipe_error():
            log_exception(self, exc_info)

    BaseServer.handle_error = my_handle_error
    handlers.BaseHandler.log_exception = my_log_exception

patch_broken_pipe_error()

You can also enable proxy_ignore_client_abort option in your nginx conf. In this case nginx will not break connection with backend if client (web-browser) does this. — San4ez, Mar 25 '14 at 06:18
This code needs to be put in settings.py - using Django 1.5. — Emil Stenström, Aug 18 '14 at 14:56

score 4 · Answer 4 · edited May 23 '17 at 12:16

Here is a way to prevent the to print the message to stderr. Just monkey patch the BaseServer.handle_error function. This is how I do it:

def patch_broken_pipe_error():
    """Monkey Patch BaseServer.handle_error to not write
    a stacktrace to stderr on broken pipe.
    https://stackoverflow.com/a/7913160"""
    import sys
    from SocketServer import BaseServer

    handle_error = BaseServer.handle_error

    def my_handle_error(self, request, client_address):
        type, err, tb = sys.exc_info()
        # there might be better ways to detect the specific erro
        if repr(err) == "error(32, 'Broken pipe')":
            # you may ignore it...
            logging.getLogger('mylog').warn(err)
        else:
            handle_error(self, request, client_address)

    BaseServer.handle_error = my_handle_error


patch_broken_pipe_error()

score 4 · Answer 5 · answered Oct 08 '21 at 00:03

In Django 3.2+ the "error" is an INFO reported by logging similar to:

INFO - Broken pipe from ('127.0.0.1', 59895)

@igor-katson has a good quick and dirty monkey patch. Updated for 3.2+ it can be:

# Suppress broken pipe errors
from django.core.servers.basehttp import WSGIServer
WSGIServer.handle_error = lambda *args, **kwargs: None

If you have a conditional check for being in development (runserver) mode in your settings.py, you could put this code there. It will squelch that annoying log entry.

Keep in mind, WSGIServer.handle_error also reports non-socket errors. If you had some other connectivity issue to debug this may get in the way.

score 3 · Answer 6 · answered Mar 16 '13 at 06:44

I was able to get rid of this by

proxy_buffering off;

This stops response buffering of proxied server. This leads to other issues of the back-end application being locked for long if the client is on a extremely slow connection.

To make it conditional for particular requests, use X-Accel-Buffering=no in the response header.

score 3 · Answer 7 · answered Mar 26 '13 at 23:43

I came up with a quick and dirty monkey patch (i don't know if it supresses any useful errors), that gets rid of this annoying error when using "./manage.py runserver" or running LiveServerTestCase tests.

Just insert it anywhere in your code, where you need that:

# Monkeypatch python not to print "Broken Pipe" errors to stdout.
import SocketServer
from wsgiref import handlers
SocketServer.BaseServer.handle_error = lambda *args, **kwargs: None
handlers.BaseHandler.log_exception = lambda *args, **kwargs: None

Gopi Kirupanithi · Answer 8 · 2013-09-11T07:14:58.050

1

I fixed it. If you use links i.e, anchor tag, inside the page, you have to face the "Borken Pipe" problem. Just use inside the link tag href='#'. Don't leave the href attribute blank. It will avoid that type of error.

edited Sep 11 '13 at 07:14

answered Sep 11 '13 at 07:07

Gopi Kirupanithi

53
1
7

score 1 · Answer 9 · edited Oct 13 '21 at 10:18

1

My BrokenPipe error disappears when I remove the "/" in the path in "urls.py"

urlpatterns = [
    path('', views.index)
]

edited Oct 13 '21 at 10:18

Emi OB

2,814
3
13
29

answered Oct 12 '21 at 09:08

Thuta_YeAg

11
1

score 1 · Answer 10 · answered Jul 15 '22 at 10:16

1

I turned

DEBUG = True

and added

if os.getcwd() == '/app':
    DEBUG = False

now it is working

answered Jul 15 '22 at 10:16

kürşat aslan

11
1

This is a nice answer because it provides new alternative solution. In which django version you test this solution ? is it on Django 3.x?. I mean it's a really old question (django 1.3) – Willy satrio nugroho Jul 17 '22 at 04:40
1

I tested with django version 4.0.4 – kürşat aslan Jul 18 '22 at 06:51

score 0 · Answer 11 · answered Mar 25 '13 at 20:46

I came across this issue as well while using tilelite. It's actually caused by a known, and now fixed, bug in python. You can resolve this issue by applying the following patch:

http://bugs.python.org/issue14574

Otherwise, you can download one of the more recent builds of python.

score 0 · Answer 12 · answered May 08 '22 at 06:19

Now currently it should be (in settings.py):

# Miscellaneous patch code

def patch_broken_pipe_error():
    """Monkey Patch BaseServer.handle_error to not write
    a stacktrace to stderr on broken pipe.
    http://stackoverflow.com/a/22618740/362702"""
    import sys
    from socketserver import BaseServer
    from wsgiref import handlers

    handle_error = BaseServer.handle_error
    log_exception = handlers.BaseHandler.log_exception

    def is_broken_pipe_error():
        type, err, tb = sys.exc_info()
        return repr(err) == "error(32, 'Broken pipe')"

    def my_handle_error(self, request, client_address):
        if not is_broken_pipe_error():
            handle_error(self, request, client_address)

    def my_log_exception(self, exc_info):
        if not is_broken_pipe_error():
            log_exception(self, exc_info)

    BaseServer = my_handle_error
    handlers.BaseHandler.log_exception = my_log_exception

patch_broken_pipe_error()

Note the lowercase socketserver import line.

Should be `BaseServer.handle_error = my_handle_error` – Maciek D. Dec 20 '22 at 09:22 — Maciek D., Dec 20 '22 at 09:22

Django Broken pipe in Debug mode

12 Answers12

Linked

Related