Python: recursive function to find value in nested dictionary keeps returning 'None'

Question

I have a nested dictionary containing microscopy acquisition metadata that would look like:

metadata_dict = {Metadata:   {"_text"       :   "\n\n  ",
                              "Core"        :   [{"_text"   :   \n\n  ",
                                                  "Guid"        :   [{"_text"   :   "c..."}],
                                                  "UserID"      :   [{"_text"   :   "xyz"}]
                                                  "AppSW"       :   [{"_text"   :   "xT"}]
                                                  "AppSWVer"    :   [{"_text"   :   "0"}]
                                                }],
                              "Instrument"  :   [{"_text"   :   "\n\n  ",
                                                  "ContrSWVer   :   [{"_text":  :   "10..."}]
                                                   ...

and so forth (hope the indentations make it a bit less chaotic). I only require certain key/value pairs and I'm using python to get the values to specifiable keys. The .get() method returns 'None' no matter the argument, so I tried the following to go over all values in the dictionary to find specific keys:

def lookup(dic,prop):
    for k, v in dic.items():
        if k == prop:
            if not isinstance(v, dict):
                return v
            else:
                for key, value in v:
                    return value
        elif isinstance(v, dict):
            lookup(v, prop)

Calling this method, e. g. by

UserID = lookup(metadata_dict, 'UserID')
print(UserID)

I only get 'None' outputs regardless of the level at which the argument is nested in the dictionary.

This post seemed to tackle the same problem, however, after changing the last two lines to

        elif isinstance(v, dict):
            return lookup(v, prop)

I still get but 'None' returns. Could this be caused by recursively calling the method with the second argument being prop resulting in the method running with v, prop (and prop being an undefined variable) rather than inheriting the argument ('UserID' in the example) from the initial function? If so, why wouldn't this return an error, considering prop is undefined?

score 2 · Accepted Answer · answered Feb 16 '22 at 07:55

The issue is UserID is inside ['Metadata']['Core'] which is a list.

You are only recursing if v is a dict.

You could modify your code to handle that:

def lookup(dic, prop):
    for k, v in dic.items():
        if k == prop:
            if not isinstance(v, dict):
                return v
        if isinstance(v, dict):
            return lookup(v, prop)
        if isinstance(v, list):
            for item in v:
                found = lookup(item, prop)
                if found:
                    return found

>>> lookup(metadata_dict, "UserID")
[{'_text': 'xyz'}]

I left out:

else:
    for key, value in v:
        return value

v is a dict, so this should raise an error. Also return will exit the function - so this will only ever process a single value.

If there can be multiple matches - you could use a generator instead.

def lookup(dic, prop):
    for k, v in dic.items():
        if k == prop:
            if not isinstance(v, dict):
                yield v
        if isinstance(v, dict):
            yield from lookup(v, prop)
        if isinstance(v, list):
            for item in v:
                yield from lookup(item, prop)

Output:

>>> for result in lookup(metadata_dict, "UserID"):
...     print(result)
[{'_text': 'xyz'}]

Example with multiple values:

>>> for result in lookup({"Foo": [{"UserID": 1}], "Bar": {"Baz":[{"Hi": {"UserID": "2"}}]}}, "UserID"):
...     print(result)
1
2

Thank you, that makes a lot of sense (and was hard to miss in hindsight). The code works just fine. — Tim, Feb 16 '22 at 08:14

score 0 · Answer 2 · answered Feb 16 '22 at 07:41

You can try this to see if it works, if not, i suggest that you explain the results you want, you can use data as an example

def lookup(dic, prop):
    res = []
    for item in dic.values():
        if isinstance(item, list):
            for sub_item in item:
                for k, v in sub_item.items():
                    if k == prop:
                        res.append(v)
        elif isinstance(item, dict):
            for k, v in item.items():
                if k == prop:
                    res.append(v)
    return res


metadata_dict = {"Metadata":   {"_text"       :   "\n\n  ",
                              "Core"        :   [{"_text"   :   "\n\n  ",
                                                  "Guid"        :   [{"_text"   :   "c..."}],
                                                  "UserID"      :   [{"_text"   :   "xyz"}],
                                                  "AppSW"       :   [{"_text"   :   "xT"}],
                                                  "AppSWVer"    :   [{"_text"   :   "0"}]
                                                }],
                              "Instrument"  :   [{"_text"   :   "\n\n  ",
                                                  "Guid"        :   [{"_text"   :   "124"}],
                                                  "UserID"      :   [{"_text"   :   "waefe"}],
                                                  "AppSW"       :   [{"_text"   :   "asfa"}],
                                                  "AppSWVer"    :   [{"_text"   :   "21"}]
                                                }]
}}

print(lookup(metadata_dict["Metadata"], "UserID"))

Python: recursive function to find value in nested dictionary keeps returning 'None'

2 Answers2