Unicode display issues (UnicodeEncodeError) via "heroku run python"

Question

If I connect to a Heroku worker dyno with heroku run (e.g. heroku run python for an interactive Python session), any attempt to display Unicode characters via this results in a UnicodeEncodeError

Locally:

$ python
Python 2.7.1 (r271:86832, Jun 25 2011, 05:09:01) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print u'\xa3'
£

Via heroku run:

$ heroku run python
Running python attached to terminal... up, run.1
Python 2.7.2 (default, Oct 31 2011, 16:22:04) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print u'\xa3'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 0: ordinal not in range(128)
>>>

Now, if I heroku run bash and use echo to try to display things, all seems fine (except my local choice of font!):

$ heroku run bash
Running bash attached to terminal... up, run.2
~ $ echo -e "\xa3"
?

I assume I'm doing something wrong / missing something, but somewhat lost as to what, or further lines of investigation.

score 8 · Accepted Answer · edited Dec 15 '19 at 22:19

8

I believe this will fix your problem:

$ heroku config:set LANG=en_US.UTF-8

or, if after you try to run that, heroku complains that you need to supply the app parameter:

$ heroku config:set LANG=en_US.UTF-8 -a <app_name>

edited Dec 15 '19 at 22:19

Will Ediger

893
9
17

answered Mar 29 '12 at 00:37

Kenneth Reitz

8,559
4
31
34

This is default on all new Heroku Python apps now :) – Kenneth Reitz Apr 01 '12 at 16:47
Oh, as a function of the buildpack? – Kristian Glass Apr 01 '12 at 19:44
This helped so much. Thanks a lot. – Anthony Dito Jun 18 '16 at 17:09

score 3 · Answer 2 · edited May 23 '17 at 12:33

3

None of the above worked but this did:

heroku config:add PYTHONIOENCODING=utf8

This is one of the answers to this question here:

UnicodeEncodeError only when running as a cron job

unfortunately it wasn't the accepted answer and that question is marked as a dup so I'm reposting here.

edited May 23 '17 at 12:33

Community

1
1

answered Jan 02 '14 at 19:08

Andy Baker

21,158
12
58
71

score 3 · Answer 3 · answered Mar 28 '12 at 13:42

3

I had similar issues. The problem appears to be that, when it is not outputting to a terminal, Python 2 chooses ascii as the default encoding for the print statement. There are ways around this, including the complex but accurate method described here. There are similar methods scattered around the net.

However, there is an easier, but deprecated, solution. This happens to be the one I use, which consists of inserting this at the beginning of your program:

reload(sys) 
sys.setdefaultencoding("utf-8")

This forces print to use the utf-8 default encoding rather than ascii, and should, hopefully, fix your problem.

answered Mar 28 '12 at 13:42

ShankarG

1,105
11
26

Alas the solution you describe doesn't seem to work for me, but the link seems highly relevant, thanks! – Kristian Glass Mar 28 '12 at 13:45
It may not work in the interpreter - I am away from my system and not able to check - but it ought to work if you try it in a script? – ShankarG Mar 28 '12 at 15:25

Unicode display issues (UnicodeEncodeError) via "heroku run python"

3 Answers3