20

Is there an easy way of iterating over nested dictionary, which may consist of other objects like lists, tuples, then again dictionaries so that iteration covers all the elements of these other objects?

For example, if I type a key of a nested dictionary object, I would get it all listed in the Python interpreter.

[edit] here is example dictionary:

{
'key_1': 'value_1',
'key_2': {'key_21': [(2100, 2101), (2110, 2111)],
      'key_22': ['l1', 'l2'],
      'key_23': {'key_231': 'v'},
      'key_24': {'key_241': 502,
             'key_242': [(5, 0), (7, 0)],
             'key_243': {'key_2431': [0, 0],
                 'key_2432': 504,
                 'key_2433': [(11451, 0), (11452, 0)]
                },
             'key_244': {'key_2441': {'key_24411': {'key_244111': 'v_24411',
                                'key_244112': [(5549, 0)]
                               },
                          'key_24412':'v_24412'
                         },
                 'key_2441': ['ll1', 'll2']
                }
            },
     }
}

sorry for being unreadable, but I did the best that I could.

Hans Goldman
  • 564
  • 3
  • 12
theta
  • 24,593
  • 37
  • 119
  • 159
  • For the second part of your question, you might want [a pretty printer](http://www.doughellmann.com/PyMOTW/pprint/). – miku Dec 01 '11 at 00:37
  • 2
    Why down-voting without providing reason for it? Is question badly asked? Is dictionary object example bad? It's just part of much larger dictionary for which I want to use graphviz in automated manner if possible without crawling by hand – theta Dec 01 '11 at 01:22
  • 2
    well, some people are just trigger happy when they see other people attempting to iterate through dictionaries or hash tables – prusswan Dec 23 '11 at 02:30
  • Why isn't there an accepted answer? Graddy's and @NeilenMarais solutions work well. – zelusp Aug 08 '16 at 23:09
  • 1
    @zelusp: http://code.activestate.com/recipes/577982-recursively-walk-python-objects/ – theta Aug 09 '16 at 20:50
  • @theta how'd you get that code to give you a tuple of keys and a list of values instead of a list of keys and each value separately? – FancyDolphin Sep 07 '16 at 08:24

6 Answers6

19
def recurse(d):
  if type(d)==type({}):
    for k in d:
      recurse(d[k])
  else:
    print d
Thomas Fenzl
  • 4,342
  • 1
  • 17
  • 25
Graddy
  • 2,828
  • 5
  • 19
  • 25
  • 2
    @theta: works fine, for one interpretation of your question. If you want it to loop through lists as well, add an `elif isinstance([], (list, tuple)):` and then `for v in d: recurse(v)`. – naught101 Aug 21 '15 at 03:07
  • 1
    NOTE : Recurssive approach fails in case of large nested dicts... Gives error : maximum recursion depth exceeded. So it depends upon your requirement. Use iterative one in case of large nested dicts problems. – MANU Jan 13 '17 at 11:04
7

A generator version of Graddy's recurse() answer above that should not explode on strings, and also gives you the compound key (cookie crumb trail?) showing how you arrived at a certain value:

def recurse(d, keys=()):
    if type(d) == dict:
         for k in d:
            for rv in recurse(d[k], keys + (k, )):
                yield rv
    else:
        yield (keys, d)

for compound_key, val in recurse(eg_dict):
    print '{}: {}'.format(compound_key, val)

produces output (using the example dictionary provided in the question):

('key_1',): value_1
('key_2', 'key_21'): [(2100, 2101), (2110, 2111)]
('key_2', 'key_22'): ['l1', 'l2']
('key_2', 'key_23', 'key_231'): v
('key_2', 'key_24', 'key_241'): 502
('key_2', 'key_24', 'key_243', 'key_2433'): [(11451, 0), (11452, 0)]
('key_2', 'key_24', 'key_243', 'key_2432'): 504
('key_2', 'key_24', 'key_243', 'key_2431'): [0, 0]
('key_2', 'key_24', 'key_242'): [(5, 0), (7, 0)]
('key_2', 'key_24', 'key_244', 'key_2441'): ['ll1', 'll2']

In Python 3 the second yield loop should be replaceable with yield from. This generator could be made more general by replacing the type(d) == dict test with isinstance(d, collections.Mapping), using the Mapping ABC from the collections module.

NeilenMarais
  • 2,949
  • 1
  • 25
  • 23
3

Here is another solution, 

#!/usr/bin/python

d = {'key_1': 'value_1',
     'key_2': {'key_21': [(2100, 2101), (2110, 2111)],
           'key_22': ['l1', 'l2'],
           'key_23': {'key_231': 'v'},
           'key_24': {'key_241': 502,
                      'key_242': [(5, 0), (7, 0)],
                      'key_243': {'key_2431': [0, 0],
                                  'key_2432': 504,
                                  'key_2433': [(11451, 0), (11452, 0)]},
                      'key_244': {'key_2441': ['ll1', 'll2']}}}}

def search_it(nested, target):
    found = []
    for key, value in nested.iteritems():
        if key == target:
            found.append(value)
        elif isinstance(value, dict):
            found.extend(search_it(value, target))
        elif isinstance(value, list):
            for item in value:
                if isinstance(item, dict):
                    found.extend(search_it(item, target))
        else:
            if key == target:
                found.append(value)
    return found

keys = [ 'key_242', 'key_243', 'key_242', 'key_244', 'key_1' ]

for key in keys:
    f = search_it(d, key)
    print 'Key: %s, value: %s' % (key, f[0])

Output:

Key: key_242, value: [(5, 0), (7, 0)]
Key: key_243, value: {'key_2433': [(11451, 0), (11452, 0)], 'key_2432': 504, 'key_2431': 
 [0, 0]}
Key: key_242, value: [(5, 0), (7, 0)]
Key: key_244, value: {'key_2441': ['ll1', 'll2']}
Key: key_1, value: value_1
James Sapam
  • 16,036
  • 12
  • 50
  • 73
2

Iterating over a nested dictionary containing unexpected nested elements.

Here is my solution :

# d is the nested dictionary

for item in d:
    if type(item) == list:
        print "Item is a list"
        for i in item: print i
    elif type(item) == dict:
        print "Item is a dict"
        for i in item: print i
    elif type(item) == tuple:
        print "Item is a tuple"
        for i in item: print i
    else:
        print "Item is not a list, neither a dict and not even a tuple"
        print item

I think the above example is very general, you can mold it for your use case.

Yugal Jindle
  • 44,057
  • 43
  • 129
  • 197
0

I took a small portion of your dictionary

d = {
    'key_1': 'value_1',
    'key_2': {'key_21': [(2100, 2101), (2110, 2111)]},
    'key_22': ['l1', 'l2'],
    'key_23': {'key_231': 'v'}
}

With a NestedDict it is straightforward to access all values.

>>> from ndicts.ndicts import NestedDict
>>> nd = NestedDict(d)
>>> list(nd.values())
['value_1', [(2100, 2101), (2110, 2111)], ['l1', 'l2'], 'v']

If you want to flatten this list you can use this recursive function

from typing import Iterable

def flatten(li):
    def wrap(x, result):
        if isinstance(x, Iterable) and type(x) is not str:
            for i in x:
                wrap(i, result)
        else:
            result.append(x)
    result = []
    wrap(li, result)
    return result

>>> flatten(list(nd.values()))
['value_1', 2100, 2101, 2110, 2111, 'l1', 'l2', 'v']
edd313
  • 1,109
  • 7
  • 20
0

What about using a general-purpose wrapper generator, like the following:

def recursive(coll):
    """Return a generator for all atomic values in coll and its subcollections.
    An atomic value is one that's not iterable as determined by iter."""
    try:
        k = iter(coll)
        for x in k:
            for y in recursive(x):
                yield y
    except TypeError:
        yield coll


def test():
    t = [[1,2,3], 4, 5, [6, [7, 8], 9]]
    for x in recursive(t):
        print x
Edmund
  • 10,533
  • 3
  • 39
  • 57