0

I'm fetching JSON with Requests from an API (using Python 3.5) and when I'm trying to print (or use) the JSON, either by response.text, json.loads(...) or response.json(), I get an UnicodeEncodeError.

print(response.text)
UnicodeEncodeError: 'ascii' codec can't encode character '\xc5' in position 676: ordinal not in range(128)

The JSON contains an array of dictionaries with country names and some of them contain special characters, e.g.: (just one dictionary in the binary array for example)

b'[{\n "name" : "\xc3\x85land Islands"\n}]

I have no idea why there is an encoding problem and also why "ascii" is used when Requests detects an UTF-8 encoding (and even by setting it manually to UTF-8 doesn't change anything).

Edit2: The problem was Microsoft Visual Studio Code 1.4. It wasn't able to print the characters.

Andre
  • 3
  • 4
  • The error states that the character cannot be *encoded* with ascii - which means the problem is not reading/decoding the text but encoding it to bytes (when printing) – janbrohl Aug 08 '16 at 18:14
  • Possible duplicate of [UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range(128)](http://stackoverflow.com/questions/9942594/unicodeencodeerror-ascii-codec-cant-encode-character-u-xa0-in-position-20) – Nick Bull Aug 08 '16 at 18:22

1 Answers1

1

If your code is running within VS, then it sounds that Python can't work out the encoding of the inbuilt console, so defaults to ASCII. If you try to print any non-ASCII then Python throws an error rather printing text that won't display.

You can force Python's encoding by using the PYTHONIOENCODING environment variable. Set it within the run configuration for the script.

Depending on Visual Studio's console, you may get away with:

PYTHONIOENCODING=utf-8

or you may have to use a typical 8bit charset like:

PYTHONIOENCODING=windows-1252
Alastair McCormack
  • 26,573
  • 8
  • 77
  • 100
  • 1
    The `utf-8` environment variable setting did not help me - still getting unicode errors writing via logger to console in VS code 1.21 using Python3.6.4. The `windows-1252` setting created even more problems, though I'm on a Mac. – abulka Mar 12 '18 at 00:22