2

Using requests to query the DarkSky API says it returns UTF-8 encoded document, but string is defaulting to ASCII with error. If I explicitly encode as UTF-8, there are no errors, but string contains extra characters and raw unicode. What's going on? I've set my py file to use UTF-8 encoding in Sublime.

# Fetch weather data from DarkSky, parse resulting JSON
try:
url = "https://api.darksky.net/forecast/" + API_KEY + "/" + LAT + "," + LONG + "?exclude=[minutely,hourly,alerts,flags]&units=us"
response = requests.get(url)
data = response.json()
print(response.headers['content-type'])
print(response.encoding)

which returns: application/json; charset=utf-8

d_summary = data['daily']['summary']
print("Daily Summary: ", d_summary.encode('utf-8'))

which returns: Daily Summary: b'No precipitation throughout the week, with temperatures rising to 82\xc2\xb0F on Tuesday.'

What's going on with the extra characters in front and quoted substring with unicode text?

Ben
  • 21
  • 1
  • 3

1 Answers1

0

I don't see any problem here. Decoding the JSON doesn't cause an error, and encoding to UTF-8 produces a byte string literal repr b'...' as expected. Top-bit-set bytes are expected to look like \xXX in byte string literals.

string is defaulting to ASCII with error

What do you mean by that? Please show us the actual problem.

My guess is you are trying to print non-ASCII characters to the terminal on Windows and getting UnicodeEncodeError. If so that's because the Windows Console is broken and can't print Unicode properly. PEP 528 works around the problem in Python 3.6.

bobince
  • 528,062
  • 107
  • 651
  • 834
  • Using Sublime on a mac, but that could stills very likely be the problem. Running the same code in the python 3.6.3. IDE displays the unicode text and symbols just fine. First the error from Sublime: `print("Daily Summary: ", d_summary)` produces `UnicodeEncodeError: 'ascii' codec can't encode character '\xb0' in position 69: ordinal not in range(128)` printing the same d_summary in the IDE returns `No precipitation throughout the week, with temperatures bottoming out at 64°F on Thursday.` – Ben Nov 24 '17 at 17:43
  • Ah! Sounds like https://stackoverflow.com/questions/39576308/printing-utf-8-in-python-3-using-sublime-text-3 then! – bobince Nov 25 '17 at 20:56
  • Thank you so much! I'm still new to coding and I've thrown hours at this. Glad to finally understand what was going on and how to fix it! – Ben Nov 26 '17 at 18:06
  • I too experienced this problem using a mac and PyCharm IDE. Trying to print the response in the pycharm terminal gave the same error. But running through the mac terminal gave no error and displayed correctly. – DJ319 May 10 '18 at 11:49