Check if key exists and iterate the JSON array using Python

Question

I have a bunch of JSON data from Facebook posts like the one below:

{"from": {"id": "8", "name": "Mary Pinter"}, "message": "How ARE you?", "comments": {"count": 0}, "updated_time": "2012-05-01", "created_time": "2012-05-01", "to": {"data": [{"id": "1543", "name": "Honey Pinter"}]}, "type": "status", "id": "id_7"}

The JSON data is semi-structured and all is not the same. Below is my code:

import json 

str = '{"from": {"id": "8", "name": "Mary Pinter"}, "message": "How ARE you?", "comments": {"count": 0}, "updated_time": "2012-05-01", "created_time": "2012-05-01", "to": {"data": [{"id": "1543", "name": "Honey Pinter"}]}, "type": "status", "id": "id_7"}'
data = json.loads(str)

post_id = data['id']
post_type = data['type']
print(post_id)
print(post_type)

created_time = data['created_time']
updated_time = data['updated_time']
print(created_time)
print(updated_time)

if data.get('application'):
    app_id = data['application'].get('id', 0)
    print(app_id)
else:
    print('null')

#if data.get('to'):
#... This is the part I am not sure how to do
# Since it is in the form "to": {"data":[{"id":...}]}

I want the code to print the to_id as 1543 else print 'null'

I am not sure how to do this.

Answer 1

import json

jsonData = """{"from": {"id": "8", "name": "Mary Pinter"}, "message": "How ARE you?", "comments": {"count": 0}, "updated_time": "2012-05-01", "created_time": "2012-05-01", "to": {"data": [{"id": "1543", "name": "Honey Pinter"}]}, "type": "status", "id": "id_7"}"""

def getTargetIds(jsonData):
    data = json.loads(jsonData)
    if 'to' not in data:
        raise ValueError("No target in given data")
    if 'data' not in data['to']:
        raise ValueError("No data for target")

    for dest in data['to']['data']:
        if 'id' not in dest:
            continue
        targetId = dest['id']
        print("to_id:", targetId)

Output:

In [9]: getTargetIds(s)
to_id: 1543

Answer 2

If all you want is to check if key exists or not

h = {'a': 1}
'b' in h # returns False

If you want to check if there is a value for key

h.get('b') # returns None

Return a default value if actual value is missing

h.get('b', 'Default value')

Answer 3

It is a good practice to create helper utility methods for things like that so that whenever you need to change the logic of attribute validation it would be in one place, and the code will be more readable for the followers.

For example create a helper method (or class JsonUtils with static methods) in json_utils.py:

def get_attribute(data, attribute, default_value):
    return data.get(attribute) or default_value

and then use it in your project:

from json_utils import get_attribute

def my_cool_iteration_func(data):

    data_to = get_attribute(data, 'to', None)
    if not data_to:
        return

    data_to_data = get_attribute(data_to, 'data', [])
    for item in data_to_data:
        print('The id is: %s' % get_attribute(item, 'id', 'null'))

IMPORTANT NOTE:

There is a reason I am using data.get(attribute) or default_value instead of simply data.get(attribute, default_value):

{'my_key': None}.get('my_key', 'nothing') # returns None
{'my_key': None}.get('my_key') or 'nothing' # returns 'nothing'

In my applications getting attribute with value 'null' is the same as not getting the attribute at all. If your usage is different, you need to change this.

Answer 4

if "my_data" in my_json_data:
         print json.dumps(my_json_data["my_data"])

Answer 5

I wrote a tiny function for this purpose. Feel free to repurpose,

def is_json_key_present(json, key):
    try:
        buf = json[key]
    except KeyError:
        return False

    return True

Answer 6

jsonData = """{"from": {"id": "8", "name": "Mary Pinter"}, "message": "How ARE you?", "comments": {"count": 0}, "updated_time": "2012-05-01", "created_time": "2012-05-01", "to": {"data": [{"id": "1543", "name": "Honey Pinter"}, {"name": "Joe Schmoe"}]}, "type": "status", "id": "id_7"}"""

def getTargetIds(jsonData):
    data = json.loads(jsonData)
    for dest in data['to']['data']:
        print("to_id:", dest.get('id', 'null'))

Try it:

>>> getTargetIds(jsonData)
to_id: 1543
to_id: null

Or, if you just want to skip over values missing ids instead of printing 'null':

def getTargetIds(jsonData):
    data = json.loads(jsonData)
    for dest in data['to']['data']:
        if 'id' in to_id:
            print("to_id:", dest['id'])

So:

>>> getTargetIds(jsonData)
to_id: 1543

Of course in real life, you probably don't want to print each id, but to store them and do something with them, but that's another issue.

Answer 7

You can use a try-except

try:
   print(str.to.id)
except AttributeError: # Not a Retweet
   print('null')