Python handling KeyError in nested JSON

Question

def get_users():
    url = "https://blablabla/api/users"
    headers = {"Authorization": accessToken, "Content-Type": "application/json", "Accept": "application/json"}
    r = requests.get(url, headers=headers)
    r_dict = r.json()
    return r_dict

get_user_function = get_users()

I get a JSON list with given request. I store the list in a global variable. I mostlikely can not store it to a file, so I have to work with the list, which contains several thousand JSON items looking like this:

[
    {
        "created": "2021-01-1 09:02:35.112 +0000 UTC",
        "id": "123456",
        "clientID": "client_client",
        "name": "name_name",
        "old": true,
        "config": {
            "config_option_1": false,
            "config_option_2": true,
            "config_option_3": false,
            "config_option_4": false,
            "config_option_5": false,
            "config_option_6": false,
            "config_option_7": false,
            "config_option_8": "123",
            "config_option_9": "456",
            "config_option_10": "",
            "config_option_11": {},
            "config_option_12": {
                "config_option_12.1": {
                    "config_option_12.1.1": true,
                    "config_option_12.1.2": true,
                    "config_option_12.1.3": false,
                    "config_option_12.1.4": true,
                    "config_option_12.1.5": false,
                    "config_option_12.1.6": false,
                    "config_option_12.1.7": false,
                    "config_option_12.1.8": false
                }}}}]

This is about half of the very first item. No I iterate through that list, to store the "name" to an empty dict with several "config_options" attached to that name. Problem is: not all of them items are the same, so sometimes there is "created" missing, or "config_option_12" is empty or located at a different spot. This leads to a KeyError. I used exceptions to just ignore those cases, but I can't just ignore them. I need to either find the "config_option" that is located unter antother parent or in case something is missing, just leave it empty.

I did it like this:

my_dict = {}

for i in range(len(get_user_function)):
  try:
    item = get_user_function[i]
    my_dict[i] = {item["name"]: [item["id"], item["created"], item["config"]["config_option_12"]] for item in get_user_function}
  except KeyError:
    continue 
print(my_dict)

score 1 · Answer 1 · answered Nov 07 '22 at 10:15

1

Instead of using brackets you can use dict.get function which does not throw KeyErrror but instead assigns None or a default value you pass as second argument.

created = item.get("created", "2020-01-01 00:00:00.000 +0000 UTC")

answered Nov 07 '22 at 10:15

treuss

1,913
1
15

c8999c 3f964f64 · Answer 2 · 2022-11-07T13:23:04.543

I was dealing with the same problem of nested dictionaries with optional fields, and I got a couple of different answers here:

Dealing with optional python dictionary fields

using my_dict.get() instead of my_dict[] - yes, but: the problem is that json.get("optional").get("maybe") will throw an exception if the optional field is missing, because you will be calling .get() on a None object (which is the default return value when .get() can't find the field)

I liked giving a default argument my_dict.get("optional", {}).get("maybe"), this prevents the None object problem, but there are different solutions like checking if the optional field exists in the json in the first place, etc...

I must say, it didnt seem like there was one obvious solution to this, so use whatever you want.

Python handling KeyError in nested JSON

2 Answers2