json KeyError with json.loads

Question

JSON seems to be hiccuping on the following statements:

{"delete":{"status":{"id":12600579001,"user_id":55389449}}}

code snippet:

temp = json.loads(line)
text = temp['text']

I get the following error output when the above code snippet encounters lines similar to the above JSON 'dictionary':

text = temp['text']
KeyError: 'text'

Is it because there is no "text" key in the line or because "delete" is not in the dictionary?

Answer 1

It looks like this happens because 'text' is not in there. Maybe you could use something like

'text' in temp

to check that 'text' exists before trying to use it.

Edit:

I've taken the example given in the comment and added a if/elif/else block to it.

#! /usr/bin/python
import sys
import json
f = open(sys.argv[1])
for line in f:
    j = json.loads(line)
    try:
        if 'text' in j:
            print 'TEXT: ', j['text']
        elif 'delete' in j:
            print 'DELETE: ', j['delete']
        else:
            print 'Everything: ', j
    except: 
        print "EXCEPTION: ", j

Sample Chunk #1:

{u'favorited': False, u'contributors': None, u'truncated': False, u'text': ---- snip ---- }

Sample Chunk #2:

{u'delete': {u'status': {u'user_id': 55389449, u'id': 12600579001L}}}

Answer 2

use dict.get(key[, default]) if there is a valid case when the key is missing: temp.get('text') instead of temp['text'] won’t throw an exception but return the null value None if the key is not found.

EAFP (Easier to Ask for Forgiveness than Permission) is more Pythonic than LBYL (Look Before You Leap).

Answer 3

Is it because there is no "text" key in the line or because "delete" is not in the dictionary?

It's because there is no "text" key. If you print temp or check whether the key 'text' is in the resulting Python dictionary, you'll notice that there is no key named 'text'. In fact, temp only has one key: 'delete'. The dictionary that is referenced by 'delete' contains a single key 'status' that contains another dictionary with two keys: 'user_id' and 'id'.

In other words, your structure is this:

{
    "delete" : {
        "status" : {
            "id" : 12600579001,
            "user_id" : 55389449
        }
    }
}

As you can see, there is no "text" key anywhere.

Furthermore, you can check it yourself:

>>> 'text' in temp
False
>>> 'delete' in temp
True

Answer 4

Try this to see the problem in detail:

import json
line = '{"delete":{"status":{"id":12600579001,"user_id":55389449}}}'
print 'line:', line
temp = json.loads(line)
print 'temp:', json.dumps(temp, indent=4)
print 'keys in temp:', temp.keys()

Which generates this output:

line: {"delete":{"status":{"id":12600579001,"user_id":55389449}}}
temp: {
    "delete": {
        "status": {
            "user_id": 55389449, 
            "id": 12600579001
        }
    }
}
keys in temp: [u'delete']

The only key in the temp dict is 'delete'. Thus temp['text'] generates a KeyError.

Answer 5

From the snippet you posted, it looks like temp should only have one item, with the key "delete". You do not have a key 'text', so I am not sure what temp['text'] should look up.

Answer 6

try it like this:

    temp = json.load(line)
    for lines in temp
       text = lines['text']

Answer 7

Why not put this between the first and second lines:

print temp

Answer 8

Thanks to all for the suggestions. The heart of the problem was that the Twitter json format has a dictionary within a dictionary. The solution involves a double index to get at the variables that I need to check.

Answer 9

#!/usr/bin/env python
import sys
import json
from pprint import pprint


json_file=sys.argv[1]
json_data=open(json_file)
j = json.load(json_data)

def main():
  for attribute_key in j['root_attribute']:
   try: print attribute_key['name'], attribute_key['status'], attribute_key['text']
   except KeyError: pass

if __name__ == "__main__":
    main()

Answer 10

Just an updated version of the verified answer.

If the frequency of that error happening (meaning that the variable text does not exist in the json file) is below 50% then the way to go is with the desired answer.

But if the exception is really exceptional you should use

try:
   #your code here
except KeyError:
   continue