24

I'm working on django project and I got this error email.

Stack trace

File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/core/handlers/wsgi.py", line 180, in _get_post
    self._load_post_and_files()

File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/http/__init__.py", line 379, in _load_post_and_files
    self._post, self._files = QueryDict(self.body, encoding=self._encoding), MultiValueDict()

File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/http/__init__.py", line 335, in body
    self._body = self.read()

File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/http/__init__.py", line 391, in read
    return self._stream.read(*args, **kwargs)

File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/core/handlers/wsgi.py", line 98, in read
    result = self.buffer + self._read_limited()

File "/usr/local/lib/python2.7/dist-packages/Django-1.4.3-py2.7.egg/django/core/handlers/wsgi.py", line 92, in _read_limited
    result = self.stream.read(size)

UnreadablePostError: request data read error

Why this error is happening ?
How to solve?

arulmr
  • 8,620
  • 9
  • 54
  • 69
Batnasan Byambasuren
  • 319
  • 1
  • 3
  • 8
  • 11
    These are broken requests. E.g. someone cancelled the request while posting something. you can ignore them. But you might wanna check on which page this is happening. Maybe your app is taking too much time to load something. – Bibhas Debnath Mar 21 '13 at 10:02
  • 4
    @Bibhas I once worked with the most terrible QAs whose idea of "stress testing" was holding down "F5" in a browser. A lot of these popped out. :) – dmg Mar 21 '13 at 10:07
  • :D People and their crazy logics. :P BTW, should I post that comment as answer? It doesn't really say how to solve it though. I'm also not sure how you work around broken requests. – Bibhas Debnath Mar 21 '13 at 10:10
  • Looks like a duplicate of http://stackoverflow.com/questions/3823280/ioerror-request-data-read-error/7089413 – guettli Mar 21 '13 at 10:16
  • 1
    @Bibhas Well technically it's not an actual server error. More of a client error. You could log it in a more readable way though. – dmg Mar 21 '13 at 11:23
  • We just upgraded to mod_wsgi 4.2.8 and all of these errors went away. Yay! – Cloud Artisans Sep 06 '14 at 13:53

3 Answers3

19

Why this error is happening ?

because the server is recieving a malformed request, which can happen for many reasons. someone might've canceled loading the page, someone might have a crappy internet connection that cut out, cosmic rays could have flipped a bit.

it's not something you really need to worry about until it starts happening very frequently. you might want to make a note when these errors happen and see if it's consequently on the same page or not.

How to solve?

you can't. not at this point at least. gather some more data on when this bug occurs exactly. see if you can find a way to trigger it manually.

doxin
  • 698
  • 1
  • 7
  • 22
16

With an application at scale, you'll always get the occasional cancelled request. If you're receiving 500 emails they can be quite tedious.

I wouldn't advise completely ignoring them. If UnreadablePostErrors are pouring in, then something's wrong e.g. longer response times causes users to cancel requests.

My solution was a custom filter for admin emails where you can put any logic you want.

The simplest is probably to randomly ignore 19 out of 20 UnreadablePostErrors. That way, if something goes wrong i'll still be informed, but i'll be pestered 20x less.

If you want something more fancy, I'd go @pztrick's solution.

import logging
import random
from django.http import UnreadablePostError

class ReduceUnreadablePostErrors(logging.Filter):
    def filter(self, record):
        if record.exc_info:
            exc_value = record.exc_info[1]
            if isinstance(exc_value, UnreadablePostError):
                return random.randint(1,20) % 20==0
        return True

settings.py:

'filters': {
   'reduce_unreadable_post_errors' : {
        '()' : 'path.to.your.ReduceUnreadablePostErrors'
    },

    ...

'handlers': {
    'mail_admins': {
        'level': 'ERROR',
        'filters': ['require_debug_false','reduce_unreadable_post_errors'],
        'class': 'common.utils.log.AdminEmailHandlerWithEmail'
     },

     ...
Dean
  • 782
  • 7
  • 15
  • 3
    Nice solution. Using an expiring cache key (60 seconds? I don't know what frequency.) could also be a good solution. – pztrick May 27 '16 at 14:09
  • Do I need to call this function and if yes then what parameter is required? – Harshit verma Jul 24 '20 at 05:53
  • Django will call it for you whenever a 500 error occurs, as long as you specify the filter correctly in the logging section of your `settings.py`. If you're trying to play around in development, make sure it isn't being excluded by another filter (remove the `require_debug_false` from the above example). Check the docs for more details: https://docs.djangoproject.com/en/3.0/topics/logging/ – Dean Jul 29 '20 at 11:46
0

I wrote a throttled version of this. You'll want to see the error if it's spiking, but if it's coming in every five minutes or whatever, you can ignore it. This assumes that if it came in twice in a one-minute period, you may have some kind of issue to look into.

def skip_unreadable_post(record):
    if record.exc_info:
        exc_value = record.exc_info[1]
        if isinstance(exc_value, UnreadablePostError):
            cache_key = "settings.unreadable_post_error"
            r = make_redis_interface("CACHE")
            if r.get(cache_key) is not None:
                # We've seen this recently; let it through; hitting it a lot
                # might mean something.
                return True
            else:
                # Haven't seen this recently; cache it with a minute expiry,
                # and don't let it through.
                r.set(cache_key, "True", ex=60)
                return False
    return True

...
    "filters": {
        "skip_unreadable_posts": {
            "()": "django.utils.log.CallbackFilter",
            "callback": skip_unreadable_post,
        },
    },
...
        "django.server": {
            "level": "INFO",
            "class": "logging.StreamHandler",
            "formatter": "django.server",
            "filters": ["skip_unreadable_posts"],
        },
mlissner
  • 17,359
  • 18
  • 106
  • 169