1

I can't seem to get the unicode representation of utf-8 characters on a production server, although it works well the development workstation.

When I use Python 3.6.9 [GCC 8.3.0] on linux (Ubuntu server).

>>> str('\u0394')
'\u0394'

>>>print('\u0394')               
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0394' in position 0: ordinal not in range(256)

When I use Python 3.6.5 [GCC 4.2.1] on darwin (Mac OS)

>>> str('\u0394')
'Δ'

>>> print('\u0394') 
Δ

By the look of it, I'd say that there is something broken on the ubuntu server as str() is suppose to convert to unicode in Python 3. However, it could well be that things work differently in revision 3.6.9 w.r.t. 3.6.5. Any tips on how could I get to the bottom of this?

gtangil
  • 717
  • 7
  • 7
  • 3
    You're missing a character set to display the character you want. It's a matter of either configuring your Ubuntu system and/or installing characters set(s) you need - more likely just installing ttf fonts package(s) will fix it. Maybe this will help: https://askubuntu.com/questions/689607/how-do-i-resolve-missing-characters-in-my-unicode-font – Todd Apr 24 '20 at 17:20
  • 1
    str('\u0394') and '\u0394' are exactly the same. Your *viewer* (terminal) is incapable to display the Unicode character, supporting only the Latin1 character set. – MisterMiyagi Apr 24 '20 at 17:28
  • Installing the ttf fonts didn't help, but your comment about the character set in ubuntu led me to the fix as described bellow. Thanks @Todd! – gtangil Apr 25 '20 at 05:17
  • Awesome. Glad you found a fix. You're welcome. Maybe an update to the title of the question will help others find this who run into this on their Ubuntu distros. – Todd Apr 25 '20 at 05:17

1 Answers1

0

I found a fix amending the locale to match .UTF-8 as described in here:

export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
sudo dpkg-reconfigure locales

Previously /etc/default/locale only had en_US. I have changed it in the locale file as well, but since I can not reboot at the moment, the commands above are the best fix for me.

gtangil
  • 717
  • 7
  • 7