Compare two nested dictionary keys

Question

I need to compare only the keys of two nested dictionaries. The primary usage is for the live tests of external API responses to prevent response change.

For example, this two dictionary matched, however their values differ:

EDIT:‌ this is a sample and the actual dictionaries have dynamic keys, probably larger, and consists of integers, strings, and boolean

dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
dict2 = {"guid": {"id": {"addr": "bar", "creation_num": "2"}}}

I try to do this by resetting the values of dictionaries with this method:

def reset_values(dictionary, reset_value=0):
    for key, value in dictionary.items():
        if type(value) is dict:
            dictionary[key] = reset_values(dictionary[key], reset_value)
        else:
            dictionary[key] = reset_value
    return dictionary

This method works, but is there a more Pythonic and straightforward way?

What does the comparison of two `dict` objects have to do with changing the non-dict values in a `dict`? — chepner, Dec 06 '22 at 17:48
The values in the dictionaries are all strings. Why are you using an integer to _"reset"_ them? — accdias, Dec 06 '22 at 18:03
@chepner existing method compares them based on key/values, I need to compare the structure and keys only — Bheid, Dec 07 '22 at 08:52
@accdias it is just a sample and the actual dictionary is larger and consists of integers, strings, and boolean — Bheid, Dec 07 '22 at 08:54

rhurwitz · Answer 1 · 2022-12-08T04:36:07.650

EDIT

@Bheid is correct that by flattening the key list my solution would get tripped up when two dictionaries have the same keys but at different nesting levels. Any easy fix is to change this line:

if isinstance(v, dict):
    klist.extend(get_keys(v))

to:

if isinstance(v, dict):
    klist.append(get_keys(v))

Same idea, but the edited version preserves nested key levels.

If I understand the problem you are trying to solve it is to the compare keys of the two dictionaries (as well as subkeys of nested dictionaries) irrespective of the associated values. If two dictionaries have the same keys (and subkeys) then they are the "same" for your purposes. If that problem statement is correct then generating an in-order list of keys/sub-keys for one dictionary and comparing that list to the same for a second dictionary should be sufficient for your purposes:

dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
dict2 = {"guid": {"id": {"addr": "bar", "creation_num": "2"}}}

def get_keys(d):
    klist = []
    for k, v in d.items():
        klist.append(k)
        if isinstance(v, dict):
            klist.extend(get_keys(v))
            
    return klist

print(get_keys(dict1) == get_keys(dict2))

Output:

True

Yes you are correct, the only problem with this solution is ignoring nesting, for example, it would not detect the difference between this two dict: `{'a':0, 'b':1} , {'a':{'b':1}}` — Bheid, Dec 07 '22 at 09:11
It should probably be `klist.append(get_keys(v))` to represent the nesting levels — Tomerikoo, Dec 07 '22 at 09:16
@Tomerikoo, Thanks, we have made progress, but it only supports two levels and flattens more than that — Bheid, Dec 07 '22 at 09:28
@Bheid Why? For `dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}, "foo": "bar"}` I get as output `['guid', ['id', ['addr', 'creation_num']], 'foo']` — Tomerikoo, Dec 07 '22 at 10:40

score 0 · Answer 2 · edited Jan 04 '23 at 19:19

Disclaimer: I am the author of the ndicts package recommended in this answer.

Answer

You can use my package ndicts to do that. To install it:

pip install ndicts

Then simply convert your dictionaries into NestedDict:

from ndicts import NestedDict

dict1 = {"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
dict2 = {"guid": {"id": {"addr": "bar", "creation_num": "2"}}}
nd1 = NestedDict(dict1)
nd2 = NestedDict(dict2)

You can then check if the keys are equal:

>>> nd1.keys() == nd2.keys()
True

Here is what the keys of a NestedDict are:

>>> for k in nd1.keys():
...     print(k)
('guid', 'id', 'addr')
('guid', 'id', 'creation_num')

Comment to your solution

Your method works but as a potentially unwanted side effect it will modify the input dictionary:

>>> dict1
{"guid": {"id": {"addr": "foo", "creation_num": "4"}}}
>>> reset_values(dict1)
{'guid': {'id': {'addr': 0, 'creation_num': 0}}}
>>> dict1
{'guid': {'id': {'addr': 0, 'creation_num': 0}}}

This can be easily fixed by deepcopying the input dictionary at the beginning of your function.

You are using recursion, which is fine. However, you can also use a for loop or reduce which have some advantages. See answer to "Best way to get nested dictionary items".

One last thing, I would aim at creating a function that compares keys directly or that returns all the keys in an iterable, rather than resetting the values of your dictionaries and then comparing them.

score 0 · Answer 3 · answered Jan 03 '23 at 18:13

You can solve this by creating a dictionary hash of keys. This solution also takes into account different key ordering.

import hashlib

def has_exact_keys(d: dict, z: dict) -> bool:
    return dict_keys_hash(d) == dict_keys_hash(z)

def dict_keys_hash(d: dict) -> str:
    hash = hashlib.md5() # switch to different hashing algorithm if you want 
    for k in sorted(d.keys()):
        if isinstance(d[k], dict):
            hash.update(dict_hash(d[k]).encode('utf-8'))
        hash.update(k.encode('utf-8'))
    return hash.hexdigest()

Compare two nested dictionary keys

3 Answers3

Answer

Comment to your solution