UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 0: ordinal not in range(128)

Question

I am building a web application using Flask and Google App Engine. One of the pages in this web application makes a call via YouTube APIs to get videos given a search term.

I get the following error when I try to query YoutubeVids.html.

This only happens when when I pass a certain parameter via Jinja2 templates to the page.

file "/Users/xxxxx/App-Engine/src/templates/YoutubeVids.html", line 1, in top-level template code
    {% extends "master.html" %}
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 0: ordinal not in range(128)

INFO     2014-01-27 22:39:40,963 module.py:612] default: "GET /xxx/yyyy HTTP/1.1" 500 291

Vinay Joseph · Accepted Answer · 2014-12-13T05:58:05.293

93

Figured it out.

I put the following at the start of my python file

import sys
reload(sys)
sys.setdefaultencoding("utf-8")

edited Dec 13 '14 at 05:58

answered Jan 31 '14 at 00:34

Vinay Joseph

5,515
10
54
94

2

Or you can use: from __future__ import unicode_literals at the start. – voscausa Feb 02 '14 at 11:52
No... don't... why would you do that? – Matt Nordhoff Apr 26 '15 at 15:21
@Matt Nordhoff: Why not? I've been doing that forever (and the above give an error). – orome Oct 31 '15 at 15:15
1

@raxacoricofallapatorius: See [Dangers of sys.setdefaultencoding('utf-8')](http://stackoverflow.com/questions/28657010/dangers-of-sys-setdefaultencodingutf-8) – idbrii Dec 19 '15 at 15:45
2

@idbrii: Yes. It seems there's no way to win (except perhaps Python 3?)! – orome Dec 19 '15 at 16:10
What is the default encoding? – tomfriwel Jul 03 '17 at 03:17

score 12 · Answer 2 · answered Jan 28 '14 at 01:30

12

From the docs: Jinja2 is using Unicode internally which means that you have to pass Unicode objects to the render function or bytestrings that only consist of ASCII characters.

A normal string in Python 2.x is a bytestring. To make it unicode use:

byte_string = 'a Python string which contains non-ascii data like €äãü'
unicode_string = byte_string.decode('utf-8')

More: http://blog.notdot.net/2010/07/Getting-unicode-right-in-Python

answered Jan 28 '14 at 01:30

voscausa

11,253
2
39
67

2

Also, [Pragmatic Unicode](http://nedbatchelder.com/text/unipain.html) by Ned Batchelder, [Flask's Unicode docs](http://flask.pocoo.org/docs/dev/unicode/) and [Jinja's Unicode docs](http://jinja.pocoo.org/docs/dev/api/#unicode). – Matt Nordhoff Apr 26 '15 at 16:19

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 0: ordinal not in range(128)

2 Answers2

Linked