Loading JSON object in Python using urllib.request and json modules

Question

I am having problems making the modules 'json' and 'urllib.request' work together in a simple Python script test. Using Python 3.5 and here is the code:

import json
import urllib.request

urlData = "http://api.openweathermap.org/data/2.5/weather?q=Boras,SE"
webURL = urllib.request.urlopen(urlData)
print(webURL.read())
JSON_object = json.loads(webURL.read()) #this is the line that doesn't work

When running script through command line the error I am getting is "TypeError:the JSON object must be str, not 'bytes'". I am new to Python so there is most likely a very easy solution to is. Appreciate any help here.

Answer 1

Apart from forgetting to decode, you can only read the response once. Having called .read() already, the second call returns an empty string.

Call .read() just once, and decode the data to a string:

data = webURL.read()
print(data)
encoding = webURL.info().get_content_charset('utf-8')
JSON_object = json.loads(data.decode(encoding))

The response.info().get_content_charset() call tells you what characterset the server thinks is used.

Demo:

>>> import json
>>> import urllib.request
>>> urlData = "http://api.openweathermap.org/data/2.5/weather?q=Boras,SE"
>>> webURL = urllib.request.urlopen(urlData)
>>> data = webURL.read()
>>> encoding = webURL.info().get_content_charset('utf-8')
>>> json.loads(data.decode(encoding))
{'coord': {'lat': 57.72, 'lon': 12.94}, 'visibility': 10000, 'name': 'Boras', 'main': {'pressure': 1021, 'humidity': 71, 'temp_min': 285.15, 'temp': 286.39, 'temp_max': 288.15}, 'id': 2720501, 'weather': [{'id': 802, 'description': 'scattered clouds', 'icon': '03d', 'main': 'Clouds'}], 'wind': {'speed': 5.1, 'deg': 260}, 'sys': {'type': 1, 'country': 'SE', 'sunrise': 1443243685, 'id': 5384, 'message': 0.0132, 'sunset': 1443286590}, 'dt': 1443257400, 'cod': 200, 'base': 'stations', 'clouds': {'all': 40}}

Answer 2

As I study myself you just need to use decode('utf-8') function, then after use json.load() function to extract into json format.

>>> import json
>>> import urllib.request

>>> urlData = "http://api.openweathermap.org/data/2.5/weather?q=Boras,SE"
>>> webURL = urllib.request.urlopen(urlData)
>>> data = webURL.read()
>>> JSON_object = json.loads(data.decode('utf-8'))
{'coord': {'lat': 57.72, 'lon': 12.94}, 'visibility': 10000, 'name': 'Boras', 'main': {'pressure': 1021, 'humidity': 71, 'temp_min': 285.15, 'temp': 286.39, 'temp_max': 288.15}, 'id': 2720501, 'weather': [{'id': 802, 'description': 'scattered clouds', 'icon': '03d', 'main': 'Clouds'}], 'wind': {'speed': 5.1, 'deg': 260}, 'sys': {'type': 1, 'country': 'SE', 'sunrise': 1443243685, 'id': 5384, 'message': 0.0132, 'sunset': 1443286590}, 'dt': 1443257400, 'cod': 200, 'base': 'stations', 'clouds': {'all': 40}}

Answer 3

with an evalstr:

   println('openurl: '+ 
   eng.EvalStr('json.loads(urllib.request.urlopen("'+
                           urlData+'").read().decode("utf-8"))'));