Get path of all the elements in dictionary

Question

I have a dictionary in python and want to get path of values param1 or param2, ... to replace these with some other values. Say to get the value of param1 I need to get it like ['OptionSettings'][2]['Value']. Can I have some generic code to do this, which will print the path of all my nodes/leaves below is the dictionary

{
        "ApplicationName": "Test",
        "EnvironmentName": "ABC-Nodejs",
        "CNAMEPrefix": "ABC-Neptune",
        "SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.1 running Node.js",
        "OptionSettings": [
                           {
                            "Namespace": "aws:ec2:vpc",
                            "OptionName": "AssociatePublicIpAddress",
                            "Value": "true"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:environment",
                            "OptionName": "EnvironmentType",
                            "Value": "LoadBalanced"
                            },
                           {
                            "Namespace": "aws:ec2:vpc",
                            "OptionName": "Subnets",
                            "Value": "param1"
                            },
                           {
                            "Namespace": "aws:autoscaling:launchconfiguration",
                            "OptionName": "SecurityGroups",
                            "Value": "param2"
                            },
                           {
                            "Namespace": "aws:autoscaling:asg",
                            "OptionName": "MinSize",
                            "Value": "1"
                            },
                           {
                            "Namespace": "aws:autoscaling:asg",
                            "OptionName": "MaxSize",
                            "Value": "4"
                            },
                           {
                            "Namespace": "aws:autoscaling:asg",
                            "OptionName": "Availability Zones",
                            "Value": "Any"
                            },
                           {
                            "Namespace": "aws:autoscaling:asg",
                            "OptionName": "Cooldown",
                            "Value": "360"
                            },
                           {
                            "Namespace": "aws:autoscaling:launchconfiguration",
                            "OptionName": "IamInstanceProfile",
                            "Value": "NepRole"
                            },
                           {
                            "Namespace": "aws:autoscaling:launchconfiguration",
                            "OptionName": "MonitoringInterval",
                            "Value": "5 minutes"
                            },
                           {
                            "Namespace": "aws:autoscaling:launchconfiguration",
                            "OptionName": "RootVolumeType",
                            "Value": "gp2"
                            },
                           {
                            "Namespace": "aws:autoscaling:launchconfiguration",
                            "OptionName": "RootVolumeSize",
                            "Value": "10"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:sns:topics",
                            "OptionName": "Notification Endpoint",
                            "Value": "sunil.kumar2@pb.com"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:hostmanager",
                            "OptionName": "LogPublicationControl",
                            "Value": "false"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:command",
                            "OptionName": "DeploymentPolicy",
                            "Value": "Rolling"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:command",
                            "OptionName": "BatchSizeType",
                            "Value": "Percentage"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:command",
                            "OptionName": "BatchSize",
                            "Value": "100"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:command",
                            "OptionName": "HealthCheckSuccessThreshold",
                            "Value": "Ok"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:command",
                            "OptionName": "IgnoreHealthCheck",
                            "Value": "false"
                            },
                           {
                            "Namespace": "aws:elasticbeanstalk:command",
                            "OptionName": "Timeout",
                            "Value": "600"
                            },
                           {
                            "Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
                            "OptionName": "RollingUpdateEnabled",
                            "Value": "false"
                            },
                           {
                            "Namespace": "aws:ec2:vpc",
                            "OptionName": "ELBSubnets",
                            "Value": "param3"
                            },
                           {
                            "Namespace": "aws:elb:loadbalancer",
                            "OptionName": "SecurityGroups",
                            "Value": "param4"
                            },
                           {
                            "Namespace": "aws:elb:loadbalancer",
                            "OptionName": "ManagedSecurityGroup",
                            "Value": "param4"
                            }
                           ]

}

Yes you can... What have you tried so far to achieve it yourself though? Per the rules, SO is not a coding service. We can help debug but not write code from scratch. — Lost, Aug 29 '16 at 04:53

score 2 · Answer 1 · edited May 23 '17 at 12:22

This is essentially the same question as this:

Print complete key path for all the values of a python nested dictionary

It is essentially the necessary code. What you are going to need to add is the handling of the lists, since you are not presenting simply a key/value structure, and that can be addressed by this question here:

How to check if a variable is a dictionary in python

Which will show you how to check for a type and modify behavior based on the data type. The easiest way to accomplish this is to set up two dummy data types, one defined as an empty list and the other as an empty map and perform "isinstance" checks of the elements against the dummy types to decide on what logic you will want to execute.

Between the two of those answers, you should be able to build the paths you are looking for.

Lost is right, this is not a "write it for me" site.

score 1 · Answer 2 · answered Jul 22 '20 at 21:19

I've made a function to "flatten" dict/list nested objects.

def flatten_dict_list(obj, path):
    if isinstance(obj, (str, int, float)):
        return [(path, obj)]
    elif isinstance(obj, dict):
        ret = []
        for k, v in obj.items():
            ret.extend(flatten_dict_list(v, path + [k]))
        return ret
    elif isinstance(obj, list):            
        ret = []
        for (n, elem) in enumerate(obj):
            ret.extend(flatten_dict_list(elem, path + [str(n)]))
        return ret
    else:
        raise ValueError("supported value types: str, int, float")

You can use this function like this

>>> data = {"key1": [1, 2, 3], "key2": {"a": "A", "b": "B", "C": ["c1", "c2"]}}    
>>>  {"[" + '] ['.join(k) + ']' : v  for (k, v) in flatten_dict_list(data, [])}
{'[key1] [0]': 1,
 '[key1] [1]': 2,
 '[key1] [2]': 3,
 '[key2] [a]': 'A',
 '[key2] [b]': 'B',
 '[key2] [C] [0]': 'c1',
 '[key2] [C] [1]': 'c2'}

score -1 · Answer 3 · answered Aug 29 '16 at 04:34

-1

You could try implementing something akin to a breadth-first search or depth-first search.

answered Aug 29 '16 at 04:34

rheber

24
4

1

This answer could be very much improved. Your answer is not clear, and you don't explain or expand on anything you said. As stated above, as of now, this answer is very poor. – Christian Dean Aug 29 '16 at 04:37

score -1 · Answer 4 · answered Aug 30 '16 at 00:02

This worked for me

def dict_traverse(self,obj,path):
        if isinstance(obj, dict):
            for k,v in obj.items():
                if isinstance(v, str):
                    #if v.find('param')!=-1:
                    print(path+"["+"'"+k+"'"+"]",v)
                elif isinstance(v, list):
                    for elem in v:
                        self.myvar+=1
                        path_lst="["+"'"+k+"'"+"]"+"["+str(self.myvar)+"]"
                        self.dict_traverse(elem,path_lst)    
        elif isinstance(obj, list):
            for elem in obj:
                self.dict_traverse(elem,path)



['SolutionStackName'] 64bit Amazon Linux 2016.03 v2.1.1 running Node.js
['OptionSettings'][1]['OptionName'] AssociatePublicIpAddress
['OptionSettings'][1]['Namespace'] aws:ec2:vpc
['OptionSettings'][1]['Value'] true
['OptionSettings'][2]['OptionName'] EnvironmentType
['OptionSettings'][2]['Namespace'] aws:elasticbeanstalk:environment
['OptionSettings'][2]['Value'] LoadBalanced
['OptionSettings'][3]['OptionName'] Subnets
['OptionSettings'][3]['Namespace'] aws:ec2:vpc
['OptionSettings'][3]['Value'] param1
['OptionSettings'][4]['OptionName'] SecurityGroups
['OptionSettings'][4]['Namespace'] aws:autoscaling:launchconfiguration
['OptionSettings'][4]['Value'] param2
['OptionSettings'][5]['OptionName'] MinSize
['OptionSettings'][5]['Namespace'] aws:autoscaling:asg
['OptionSettings'][5]['Value'] 1
['OptionSettings'][6]['OptionName'] MaxSize
['OptionSettings'][6]['Namespace'] aws:autoscaling:asg
['OptionSettings'][6]['Value'] 4
['OptionSettings'][7]['OptionName'] Availability Zones
['OptionSettings'][7]['Namespace'] aws:autoscaling:asg
['OptionSettings'][7]['Value'] Any
['OptionSettings'][8]['OptionName'] Cooldown
['OptionSettings'][8]['Namespace'] aws:autoscaling:asg
['OptionSettings'][8]['Value'] 360
['OptionSettings'][9]['OptionName'] IamInstanceProfile
['OptionSettings'][9]['Namespace'] aws:autoscaling:launchconfiguration
['OptionSettings'][9]['Value'] NepRole
['OptionSettings'][10]['OptionName'] MonitoringInterval
['OptionSettings'][10]['Namespace'] aws:autoscaling:launchconfiguration
['OptionSettings'][10]['Value'] 5 minutes
['OptionSettings'][11]['OptionName'] RootVolumeType
['OptionSettings'][11]['Namespace'] aws:autoscaling:launchconfiguration
['OptionSettings'][11]['Value'] gp2
['OptionSettings'][12]['OptionName'] RootVolumeSize
['OptionSettings'][12]['Namespace'] aws:autoscaling:launchconfiguration
['OptionSettings'][12]['Value'] 10
['OptionSettings'][13]['OptionName'] Notification Endpoint
['OptionSettings'][13]['Namespace'] aws:elasticbeanstalk:sns:topics
['OptionSettings'][13]['Value'] sunil.kumar2@pb.com
['OptionSettings'][14]['OptionName'] LogPublicationControl
['OptionSettings'][14]['Namespace'] aws:elasticbeanstalk:hostmanager
['OptionSettings'][14]['Value'] false
['OptionSettings'][15]['OptionName'] DeploymentPolicy
['OptionSettings'][15]['Namespace'] aws:elasticbeanstalk:command
['OptionSettings'][15]['Value'] Rolling
['OptionSettings'][16]['OptionName'] BatchSizeType
['OptionSettings'][16]['Namespace'] aws:elasticbeanstalk:command
['OptionSettings'][16]['Value'] Percentage
['OptionSettings'][17]['OptionName'] BatchSize
['OptionSettings'][17]['Namespace'] aws:elasticbeanstalk:command
['OptionSettings'][17]['Value'] 100
['OptionSettings'][18]['OptionName'] HealthCheckSuccessThreshold
['OptionSettings'][18]['Namespace'] aws:elasticbeanstalk:command
['OptionSettings'][18]['Value'] Ok
['OptionSettings'][19]['OptionName'] IgnoreHealthCheck
['OptionSettings'][19]['Namespace'] aws:elasticbeanstalk:command
['OptionSettings'][19]['Value'] false
['OptionSettings'][20]['OptionName'] Timeout
['OptionSettings'][20]['Namespace'] aws:elasticbeanstalk:command
['OptionSettings'][20]['Value'] 600
['OptionSettings'][21]['OptionName'] RollingUpdateEnabled
['OptionSettings'][21]['Namespace'] aws:autoscaling:updatepolicy:rollingupdate
['OptionSettings'][21]['Value'] false
['OptionSettings'][22]['OptionName'] ELBSubnets
['OptionSettings'][22]['Namespace'] aws:ec2:vpc
['OptionSettings'][22]['Value'] param3
['OptionSettings'][23]['OptionName'] SecurityGroups
['OptionSettings'][23]['Namespace'] aws:elb:loadbalancer
['OptionSettings'][23]['Value'] param4
['OptionSettings'][24]['OptionName'] ManagedSecurityGroup
['OptionSettings'][24]['Namespace'] aws:elb:loadbalancer
['OptionSettings'][24]['Value'] param4
['EnvironmentName'] ABC-Nodejs
['CNAMEPrefix'] ABC-Neptune
['ApplicationName'] Test

Get path of all the elements in dictionary

4 Answers4