1

I'm using an Anaconda install of Python 3.6.3 and IPython 6.1.0 on Red Hat Enterprise Linux 7.5 . I have text files containing Unicode characters that I'm trying to work with, but I get the following error when trying to print the contents of these files:

UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f644' in position 1: ordinal not in range(128)

I believe this is due to IPython defaulting to an ASCII encoding:

In [1]: from IPython.utils.encoding import get_stream_enc; import sys

In [2]: get_stream_enc(sys.stdout)
Out[2]: 'ANSI_X3.4-1968'

I can't find anything in the IPython documentation explaining how to change this to UTF-8. Is it possible?

Tim
  • 2,756
  • 1
  • 15
  • 31
  • 1
    I'd expect that iPython is getting the encoding from the environment (`$LANG` etc). The ideal solution is to change the environment setting, or set the PYTHONIOENCODING environment variable (I discuss a similar situation is [this answer](https://stackoverflow.com/a/54599110/5320906). – snakecharmerb Mar 22 '19 at 07:13
  • 1
    @snakecharmerb Thanks! This is correct, and fixed my issue. Do you want to post this as an answer so I can accept it? – Tim Mar 22 '19 at 16:05

1 Answers1

1

iPython takes its output encoding from the environment. To use a unicode-aware encoding, either change your user's locale settings or set the PYTHONIOENCODING environment variable when calling iPython:

PYTHONIOENCODING=UTF-8 ipython
snakecharmerb
  • 47,570
  • 11
  • 100
  • 153