UnicodeDecodeError : 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

Question

On one of my machines I have the error when I am working with google apps engine or django.

For example:

app.yaml

application: demas1252c
version: 1
runtime: python
api_version: 1


handlers:
   - url: /images
static_dir: images
   - url: /css
static_dir: css
   - url: /js
static_dir: js
   - url: /.*
script: demas1252c.py

demas1252c.py

import cgi
import wsgiref.handlers


from google.appengine.ext.webapp import template
from google.appengine.ext import webapp


class MainPage(webapp.RequestHandler): 
def get(self):
values = {'id' : 10}


self.response.out.write(template.render('foto.html', values))


application = webapp.WSGIApplication([('/', MainPage)], debug = True)
wsgiref.handlers.CGIHandler().run(application)

foto.html

<!DOCTYPE html>
<html lang="en">
    <head></head>
<body>some</body>
</html>

error message:

C:\artefacts\dev\project>"c:\Program Files\Google\google_appengine\dev_appserver.py" foto-hosting
Traceback (most recent call last):
  File "c:\Program Files\Google\google_appengine\dev_appserver.py", line 69, in <module>
    run_file(__file__, globals())
  File "c:\Program Files\Google\google_appengine\dev_appserver.py", line 65, in run_file
    execfile(script_path, globals_)
  File "c:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py", line 92, in <module>
    from google.appengine.tools import dev_appserver
  File "c:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 140, in <module>
    mimetypes.add_type(mime_type, '.' + ext)
  File "C:\Python27\lib\mimetypes.py", line 344, in add_type
    init()
  File "C:\Python27\lib\mimetypes.py", line 355, in init
    db.read_windows_registry()
  File "C:\Python27\lib\mimetypes.py", line 260, in read_windows_registry
    for ctype in enum_types(mimedb):
  File "C:\Python27\lib\mimetypes.py", line 250, in enum_types
    ctype = ctype.encode(default_encoding) # omit in 3.x!
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

When I am working with static files in django (without gae) I have very similar error (with different stack).

I tried to find the reason of error and added code to mimetypes.py:

print '====='
print ctype
ctype = ctype.encode(default_encoding) # omit in 3.x!

Then I get next messages in my console:

=====
video/x-ms-wvx
=====
video/x-msvideo
=====
рєфшю/AMR
Traceback (most recent call last):

In the registry HKCR/Mime/Database/ContentType/ I have five keys with russian (cyrilic) letters. But how can I fix this error?

What is the value of "default_encoding", right now it seems that it's something that cannot convert cyrilic, using UTF-8 here should probably fix the error. — Ivo Wetzel, Nov 21 '10 at 13:15
ascii. i tryed to change it to utf-8, but I get error "UnicodeDecodeError: 'utf8' codec can't decode byte 0xef in position 0: invalid continuation byte". Anyway, I remove this keys from registry and error dissapeared. — ceth, Nov 21 '10 at 13:21
this is a bug mimetype.py changed the python code at this url http://bugs.python.org/review/9291/diff/1663/Lib/mimetypes.py — , Jan 17 '14 at 06:41

bobince · Accepted Answer · 2010-11-21T14:14:22.037

This is a bug in mimetypes, triggered by bad data in the registry. (рєфшю/AMR is not at all a valid MIME media type.)

ctype is a registry key name returned by _winreg.EnumKey, which mimetypes is expecting to be a Unicode string, but it isn't. Unlike _winreg.QueryValueEx, EnumKey returns a byte string (direct from the ANSI version of the Windows API; _winreg in Python 2 doesn't use the Unicode interfaces even though it returns Unicode strings, so it'll never read non-ANSI characters correctly).

So the attempt to .encode it fails with a UnicodeDecodeError trying to get a Unicode string before encoding it back to ASCII!

try:
    ctype = ctype.encode(default_encoding) # omit in 3.x!
except UnicodeEncodeError:
    pass

These lines in mimetypes should simply be removed.

ETA: added to bug tracker.

Thank's, this fixed my problem with IPython on Anaconda v2.1! — gaborous, Jun 23 '14 at 12:45

score 8 · Answer 2 · answered May 24 '13 at 16:25

By the way, the main culpit of the problem is QuickTime which adds non-ascii mime types to the windows registry. The easiest way to fix it is to manually find and remove from the registry the subsections of the HKCR/Mime/Database/ContentType/ starting with аудио/ and видео/.

score 5 · Answer 3 · answered Dec 08 '13 at 08:25

5

There is a patch:

http://bugs.python.org/file18143/9291.patch

Works great for me.

Just replace UnicodeEncodeError to be UnicodeError

answered Dec 08 '13 at 08:25

Wangsu

86
1
5

The accepted answer worked, I did not try this one, but I think it will work, too. – shellbye Dec 17 '15 at 12:19

score 4 · Answer 4 · answered Feb 11 '14 at 02:27

4

An alternative solution from python issue9291 by Alexandr Zarubkin (me21):

add file named sitecustomize.py in Lib\site-packages folder.

import sys
sys.setdefaultencoding("cp1251")

answered Feb 11 '14 at 02:27

HaveF

2,995
2
26
36

score 1 · Answer 5 · answered Sep 14 '12 at 11:10

1

its a python bug with latin MIME tipes in registry start regedit and inspect "HKEY_CLASSES_ROOT\MIME\Database\Content Type" for non-latin names.

answered Sep 14 '12 at 11:10

Codeboy.ru

81
1
7

UnicodeDecodeError : 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

5 Answers5

Linked