How can I directly retrieve only part of a JSON document, rather than extracting what I need from the whole thing?

Question

I want to make a JSON request with the Python library requests where I only obtain certain JSON objects.

I know that it is really easy to process the JSON object obtained to only focus in the needed information, but that would throttle the request efficiency (in case it is done repeatedly).

As said, I know this is a possibility:

url = 'www.foo.com'
r = requests.get(url).json()
#Do something with r[3]['data4'], the only one who is going to be used.

But how could I directly only obtain r[3]['data4'] from the request?

Answer 1

Short Answer

To answer your question no, you can't but to understand why you need to know what is happening behind the scenes.

Behind the scenes

When you make a request such as r = requests.get('www.foo.bar') you are making a request to the server and you are viewing the result of that request when you do r.json(). This means that you cannot just get r[3]['data'] as you are parsing what the server sends to you unless the server only sends r[3]['data']. It may be possible to filter out everything else apart from that in the response processing but I am unaware of how to do it.

Answer 2

You can't, if the server does not allow it. If the target server allows you to specify fields you want then you can send that field list in your request and server will return you only those fields in JSON. Otherwise your will have to parse full JSON response and get your desired fields.