Most effective way to parse JSON Objects

Question

I'm currently trying to access a specific value within a JSON object, I am successfully doing so using the Try / except statement.

As I am still learning Python, I am wondering if there is a more efficient way of achieving this. I often find that for different sets of JSON data the key / value pair moves, and I have to change the index value manually in my code from [0] [1] [2] or [3]

JSON data:

{
 "notifications": [
  {
   "timestamp": "1619847600000000000",
   "path_elements": [
    "Devices",
    "NME39488200",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet1",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "alignmentErrors": {
     "key": "alignmentErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "fcsErrors": {
     "key": "fcsErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "frameTooLongs": {
     "key": "frameTooLongs",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "frameTooShorts": {
     "key": "frameTooShorts",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "inErrors": {
     "key": "inErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "outDiscards": {
     "key": "outDiscards",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "outErrors": {
     "key": "outErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "symbolErrors": {
     "key": "symbolErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    }
   }
  },
  {
   "timestamp": "1622629140000000000",
   "path_elements": [
    "Devices",
    "NME39488200",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet11",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "inMulticastPkts": {
     "key": "inMulticastPkts",
     "value": {
      "avg": {
       "float": 0.03333333333333334
      },
      "max": {
       "float": 0.1
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0.047140452079103175
      },
      "weight": {
       "float": 1
      }
     }
    },
    "inOctets": {
     "key": "inOctets",
     "value": {
      "avg": {
       "float": 269348.4166666667
      },
      "max": {
       "float": 341898.5
      },
      "min": {
       "float": 223613.3
      },
      "stddev": {
       "float": 45132.897905054306
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "inUcastPkts": {
     "key": "inUcastPkts",
     "value": {
      "avg": {
       "float": 300.3333333333333
      },
      "max": {
       "float": 364
      },
      "min": {
       "float": 258.8
      },
      "stddev": {
       "float": 44.179847089921985
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "outMulticastPkts": {
     "key": "outMulticastPkts",
     "value": {
      "avg": {
       "float": 0.03333333333333334
      },
      "max": {
       "float": 0.1
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0.047140452079103175
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "outOctets": {
     "key": "outOctets",
     "value": {
      "avg": {
       "float": 28700.35
      },
      "max": {
       "float": 42799.9
      },
      "min": {
       "float": 17571.2
      },
      "stddev": {
       "float": 9737.234046475074
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "outUcastPkts": {
     "key": "outUcastPkts",
     "value": {
      "avg": {
       "float": 196.13333333333333
      },
      "max": {
       "float": 237
      },
      "min": {
       "float": 164.4
      },
      "stddev": {
       "float": 28.93957305989306
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    }
   }
  },
  {
   "timestamp": "1620649440000000000",
   "path_elements": [
    "Devices",
    "NME39488200",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet11",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "inDiscards": {
     "key": "inDiscards",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    }
   }
  },
  {
   "timestamp": "1621745580000000000",
   "path_elements": [
    "Devices",
    "JPE13031399",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet11",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "inBroadcastPkts": {
     "key": "inBroadcastPkts",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "outBroadcastPkts": {
     "key": "outBroadcastPkts",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    }
   }
  }
 ]
}

I am taking the JSON data from multiple URLs. The code I am using to extract the value 'Avg' from the Key: 'InDiscards' or 'OutDiscards' is as follows:

for url in URLs:
    JSONdata = requests.get(url, cookies=cookies, verify=False).json()
    JSONkey = JSONdata["notifications"]
    name = JSONkey[0]["path_elements"][1]
    interface = JSONkey[0]["path_elements"][5]
    try:
        outDiscards = JSONkey[0]["updates"]["outDiscards"]["value"]["avg"]["float"]
    except KeyError:
        outDiscards = JSONkey[2]["updates"]["outDiscards"]["value"]["avg"]["float"]
    try:
        inDiscards = JSONkey[0]["updates"]["inDiscards"]["value"]["avg"]["float"]
    except KeyError:
        inDiscards = JSONkey[2]["updates"]["inDiscards"]["value"]["avg"]["float"]

My understanding here is that we check index [0] for the avg value of OutDiscards (or InDiscards) , if we fail to find a value, we then check index [2] , however surely there must be a better way of achieving this given that sometimes the index may be 1/3/4/5 etc...

Does this answer your question? [How to convert JSON data into a Python object](https://stackoverflow.com/questions/6578986/how-to-convert-json-data-into-a-python-object) — JMA, Jun 02 '21 at 10:41

vinzenz · Accepted Answer · 2021-06-02T12:47:24.640

0

I recommend searching with a for loop.

First initialize your variables, I use None so I can later determine if my search routine found something

outDiscards = None
inDiscards = None

Then iterate over all elements in the notifications list

for notification in JSONdata["notifications"]:

only try to access the value when it hasn't been found yet

    if outDiscards is None:

try to to access the value and ignore the KeyError

        try:
            outDiscards = notification["updates"]["outDiscards"]["value"]["avg"]["float"]
        except KeyError:
            pass
    if inDiscards is None:
        try:
            inDiscards = notification["updates"]["inDiscards"]["value"]["avg"]["float"]
        except KeyError:
            pass

after the search routine there should be values in outDiscards and inDiscards, you can test for that with if outDiscards is not None

print(outDiscards)
print(inDiscards)

edited Jun 02 '21 at 12:47

answered Jun 02 '21 at 10:49

vinzenz

669
3
14

This worked perfectly, however i do not understand why and nor could i come to the solution myself, would it be possible to explain why this works? – netcode995 Jun 02 '21 at 11:14
what explanation do you need? should i clarify for notification in JSONdata["notifications"] ? – vinzenz Jun 02 '21 at 11:23
Regarding the = none, and then looping on that variable – netcode995 Jun 02 '21 at 12:35
i edited my answer to clarify – vinzenz Jun 02 '21 at 12:47

Most effective way to parse JSON Objects

1 Answers1