finding value from nested dictionary

Question

I have a dictionary and I need to find a value of a key. The following command does not return the expected value "mar5-deliveryreport-new"

>>> mydict['ClusterIdentifier']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'ClusterIdentifier'

>>> mydict
{u'DescribeClustersResponse': {u'DescribeClustersResult': {u'Marker': None, u'Clusters': [{u'PubliclyAccessible': True, u'MasterUsername': u'root', u'VpcSecurityGroups': [], u'ModifyStatus': None, u'NumberOfNodes': 1, u'PendingModifiedValues': {u'NodeType': None, u'ClusterType': None, u'MasterUserPassword': None, u'AutomatedSnapshotRetentionPeriod': None, u'ClusterVersion': None, u'NumberOfNodes': None}, u'VpcId': None, u'ClusterVersion': u'1.0', u'AutomatedSnapshotRetentionPeriod': 1, u'ClusterParameterGroups': [{u'ParameterApplyStatus': u'in-sync', u'ParameterGroupName': u'default.redshift-1.0'}], u'DBName': u'mydb', u'PreferredMaintenanceWindow': u'tue:08:00-tue:08:30', u'Endpoint': {u'Port': 5439, u'Address': u'mar5-deliveryreport-new.lc.us-east-1.redshift.amazonaws.com'}, u'RestoreStatus': {u'Status': u'completed', u'ProgressInMegaBytes': 804307, u'CurrentRestoreRateInMegaBytesPerSecond': 57.3072319201995, u'EstimatedTimeToCompletionInSeconds': 0, u'ElapsedTimeInSeconds': 14035, u'SnapshotSizeInMegaBytes': 804307}, u'AllowVersionUpgrade': True, u'ClusterCreateTime': 1381812358.833, u'ClusterSubnetGroupName': None, u'ClusterSecurityGroups': [{u'Status': u'active', u'ClusterSecurityGroupName': u'default'}], u'ClusterIdentifier': u'mar5-deliveryreport-new', u'AvailabilityZone': u'us-east-1a', u'NodeType': u'dw.hs1.xlarge', u'Encrypted': False, u'ClusterStatus': u'available'}]}, u'ResponseMetadata': {u'RequestId': u'233f495b-3576-11e3-83ff-d332123c25c4'}}}

>>> type(mydict)
<type 'dict'>

Your key usage is wrong. This will work for for the example you give here. mydict[''DescribeClustersResponse]['DescribeClustersResult']['Clusters'][0]['ClusterIdentifier'] — Lorcan O'Neill, Oct 15 '13 at 09:02

score 1 · Accepted Answer · answered Oct 15 '13 at 09:04

You are not getting access to it, because the way you try to access it is all wrong.

>>> import pprint
>>> pprint.pprint(mydict)
{'DescribeClustersResponse': {'DescribeClustersResult': {'Clusters': [{'AllowVersionUpgrade': True,
                                                                       'AutomatedSnapshotRetentionPeriod': 1,
                                                                       'AvailabilityZone': 'us-east-1a',
                                                                       'ClusterCreateTime': 1381812358.833,
                                                                       'ClusterIdentifier': 'mar5-deliveryreport-new',
                                                                       'ClusterParameterGroups': [{'ParameterApplyStatus': 'in-sync',
                                                                                                   'ParameterGroupName': 'default.redshift-1.0'}],
                                                                       'ClusterSecurityGroups': [{'ClusterSecurityGroupName': 'default',
                                                                                                  'Status': 'active'}],
                                                                       'ClusterStatus': 'available',
                                                                       'ClusterSubnetGroupName': None,
                                                                       'ClusterVersion': '1.0',
                                                                       'DBName': 'mydb',
                                                                       'Encrypted': False,
                                                                       'Endpoint': {'Address': 'mar5-deliveryreport-new.lc.us-east-1.redshift.amazonaws.com',
                                                                                    'Port': 5439},
                                                                       'MasterUsername': 'root',
                                                                       'ModifyStatus': None,
                                                                       'NodeType': 'dw.hs1.xlarge',
                                                                       'NumberOfNodes': 1,
                                                                       'PendingModifiedValues': {'AutomatedSnapshotRetentionPeriod': None,
                                                                                                 'ClusterType': None,
                                                                                                 'ClusterVersion': None,
                                                                                                 'MasterUserPassword': None,
                                                                                                 'NodeType': None,
                                                                                                 'NumberOfNodes': None},
                                                                       'PreferredMaintenanceWindow': 'tue:08:00-tue:08:30',
                                                                       'PubliclyAccessible': True,
                                                                       'RestoreStatus': {'CurrentRestoreRateInMegaBytesPerSecond': 57.3072319201995,
                                                                                         'ElapsedTimeInSeconds': 14035,
                                                                                         'EstimatedTimeToCompletionInSeconds': 0,
                                                                                         'ProgressInMegaBytes': 804307,
                                                                                         'SnapshotSizeInMegaBytes': 804307,
                                                                                         'Status': 'completed'},
                                                                       'VpcId': None,
                                                                       'VpcSecurityGroups': []}],
                                                         'Marker': None},
                              'ResponseMetadata': {'RequestId': '233f495b-3576-11e3-83ff-d332123c25c4'}}}
>>> mydict['DescribeClustersResponse']['DescribeClustersResult']['Clusters'][0]['ClusterIdentifier']
'mar5-deliveryreport-new'

score 1 · Answer 2 · edited May 23 '17 at 11:56

Umm... What you see is somewhat expected. There is no such key in the dictionary you are querying.

The problem is that if you know what key you search in the dictionary, you can check if it exists and retrieve it fast. However, in your case you cannot simply look for a fixed key, be cause you do not know in which sub-dictionary it is placed.

You need to traverse trough whole dict in a general case, and this can be very slow. The thing you should be looking in current state of the question is this: Finding a key recursively in a dictionary. Still some problems may arise, like what to do when there are more than one occurrences of the key in the underlying stuctures, or should you traverse lists etc...

Or you need to provide whole "path" to the dictionary you expect the key to exists E.g. mydict[u'ClusterIdentifier'][u'DescribeClustersResult'][u'Clusters'][0][u'ClusterIdentifier'].

score 1 · Answer 3 · answered Oct 15 '13 at 09:08

1

For the value of 'Address' key, you need to write it this way

myDict['DescribeClustersResponse']['DescribeClustersResult']['Clusters'][0]['Endpoint']['Address']

It will give you what you want. Hope this will help you.

answered Oct 15 '13 at 09:08

Dharmraj

576
6
26

score 1 · Answer 4 · answered Oct 15 '13 at 09:08

You'll have to adjust your expectation, because that dictionary simply does not contain that key. I'm guessing you're dealing with some JSON-RPC result or similar, and the only key present in mydict is u'DescribeClustersResponse'.

Quite often hierarchical structures like this group things for a reason. In this case, for instance, mydict[u'DescribeClustersResponse'][u'DescribeClustersResult'][u'Clusters'] is a list, suggesting that there may be more than one Cluster described. As such, even if you did have a routine to search deeply for ClusterIdentifier, you would still have to iterate through an arbitrary number of them and likely want the associated data anyway.

Also, the default print style of that dictionary was quite unreadable. pprint is your friend:

import pprint
pprint.pprint(mydict)

finding value from nested dictionary

4 Answers4