JSON object must be str, not Response

Question

I'm making a request to a URL that returns a JSON file,

response = requests.get(url)
response = json.loads(response)

And then, I am attempting to go through some of the data,

for documents in response["docs"]:
    # do something

Right now, the error I'm getting is

TypeError: the JSON object must be str, not 'Response'

In order to avoid this, I tried,

response = requests.get(url).json()

But then, I can't traverse the response because I get the error:

KeyError: 'docs'

I'm new to Python and not fully aware of the best way to get JSON data and parse it. Suggestions?

Here is a sample of the data being received,

{'status': 'OK', 'response': {'meta': {'time': 9, 'hits': 11, 'offset': 0}, 'docs': [{'type_of_material': 'News', 'pub_date': '2017-01-01T09:12:04+0000', 'document_type': 'article', '_id': '5868c7e995d0e03926078885', 'lead_paragraph': 'The country’s leader spoke proudly of the progress of its nuclear weapons and ballistic missile programs.', .....

Perhaps you want to see what is actual output of `requests.get(url).json()` is by printing it out? — metatoaster, Mar 23 '17 at 12:30
Try to print `requests.get(url).json()` and read this [answer](http://stackoverflow.com/questions/2835559/parsing-values-from-a-json-file-using-python) — McGrady, Mar 23 '17 at 12:31
Possible duplicate of [TypeError: the JSON object must be str, not 'Response' with Python 3.4](http://stackoverflow.com/questions/35291021/typeerror-the-json-object-must-be-str-not-response-with-python-3-4) — tripleee, Mar 23 '17 at 12:34
Are you sure the JSON has a `docs` key? Maybe you should post a sample of the data. — Daniel Roseman, Mar 23 '17 at 12:39
That's a Python list of dict(s), not a string of JSON. And/or the key you should be looking at is ...`['response']['docs']` — tripleee, Mar 23 '17 at 12:58

Martijn Pieters · Accepted Answer · 2020-03-07T14:35:29.403

5

You are trying to feed the response object to json.loads(). You don't get a string there, you'd have to access the .contents or .text attribute instead:

response = requests.get(url)
# Python 3
response = json.loads(response.text)
# Python 2
response = json.loads(response.content)

However, that'd be doing more work than you need to; requests supports handling JSON directly, and there is no need to import the json module:

response = requests.get(url).json()

See the JSON Response Content section of the requests Quickstart documentation.

Once loaded, you can get the doc key in the nested dictionary keyed on response:

for documents in response['response']['docs']:

edited Mar 07 '20 at 14:35

answered Mar 23 '17 at 12:31

Martijn Pieters

1,048,767
296
4,058
3,343

As I said, I've tried requests.get(url).json(), but then I don't understand how to traverse the response... – Human Cyborg Relations Mar 23 '17 at 12:35
@HumanCyborgRelations That sounds like a problem of not understanding what the API returns, not a Python problem. – chepner Mar 23 '17 at 12:45
@HumanCyborgRelations: you have a nested dictionary, the outer mapping has a `response` key, and the dictionary that references contains a `docs` key. – Martijn Pieters Mar 23 '17 at 13:27

score 0 · Answer 2 · answered Mar 23 '17 at 12:31

requests.get returns a response object, which contains status information, headers etc. You need to access the raw document text from this object in order to parse the json or just use the provided json method:

response = requests.get(url).json()

(minus error checking)

JSON object must be str, not Response

2 Answers2

Linked