3

This string:

line = '\tlong_plugin_output=\x88\\\\\x97\xe5\xff\x7f\\n'

.. When printed, produces the following output on my Macbook Pro:

>>> line = '\tlong_plugin_output=\x88\\\\\x97\xe5\xff\x7f\\n'

>>> line
'\tlong_plugin_output=\x88\\\\\x97åÿ\x7f\\n'

>>> print(line)
    long_plugin_output=\\åÿ\n

.. But then it produces this error on my ubuntu server:

>>> line = '\tlong_plugin_output=\x88\\\\\x97\xe5\xff\x7f\\n'

>>> line
'\tlong_plugin_output=\x88\\\\\x97\xe5\xff\x7f\\n'

>>> print(line)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character '\x88' in position 20: ordinal not in range(128)

The python version I'm running on my macbook:

Python 3.3.2 (v3.3.2:d047928ae3f6, May 13 2013, 13:52:24) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

$ uname -a Darwin MacBook-Pro.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

The python version I'm running on my Ubuntu server:

Python 3.2.3 (default, Sep 25 2013, 18:25:56) [GCC 4.6.3] on linux2

$ uname -a Linux net.local.net 3.2.2 #3 SMP Thu Jan 26 20:18:37 UTC 2012 i686 i686 i386 GNU/Linux

What could be causing the different behaviour on these platforms?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Raoul
  • 1,876
  • 1
  • 14
  • 14

1 Answers1

5

Python asks the terminal what encoding is being used, and encodes unicode strings to bytes when printing. Your Ubuntu server is not configured for UTF-8 display, your Mac terminal is.

See https://askubuntu.com/questions/87227/switch-encoding-of-terminal-with-a-command for help with switching your terminal locale. Any locale that can handle the specific codepoints you are trying to print is fine, but UTF8 can handle all of Unicode.

You can see what Python detected by printing sys.stdout.encoding:

>>> import sys
>>> sys.stdout.encoding
'UTF-8'
Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 1
    Thanks for a quick and spot-on answer! Indeed, my ssh session to my Ubuntu server was using a language setting that my remote server didn't understand, and this resulted in: `>>> import sys >>> sys.stdout.encoding 'ANSI_X3.4-1968'` This post was also very helpful: http://stackoverflow.com/questions/2499794/how-can-i-fix-a-locale-warning-from-perl I now have a fix client side, and server side. Thanks again Martijn! – Raoul Nov 16 '13 at 19:55