python nested dictionary comparison

Question

I'm having trouble doing a comparison against nested dictionaries in a list, I'm trying to only show the data for keys where the values changed, and the difference between them. I've tried this (example is simplified, there can be different levels of nesting in my comparison data):

old = [{"poker":{"John":{"Wins": 4, "Losses": 3}, "Jack": {"Wins": 6, "Losses": 1}}}, 
   {"Blackjack":{"Bill": {"Wins": 4, "Losses": 3}, "John": {"Wins": 7, "Losses": 0}}}]
new = [{"poker":{"John":{"Wins": 6, "Losses": 3}, "Jack": {"Wins": 6, "Losses": 5}, "Bill": {"Wins": 3, "Losses": 0}}}, 
       {"Blackjack":{"Bill": {"Wins": 4, "Losses": 3}, "John": {"Wins": 7, "Losses": 0}, "Jack": 
           {"Wins": 1, "Losses": 3}}}]


def nested_compare(new, old):
    for key in new:
        try:
            if type(new(key)) == dict:
                nested_compare(old[key], new[key])
        except:
            pass
    oldkeys = set(old.keys())
    newkeys = set(new.keys())
    samekeys = newkeys.intersection(oldkeys)
    unchanged = set(k for k in samekeys if old[k] == new[k])
    for key in list(unchanged):
        try:
            del new[key]
            del old[key]
        except KeyError:
            pass
    return new, old


final = []
x = 0
for entry in new:
    new_data, old_data  = nested_compare(entry, old[x])
    x += 1
    final.append(new_data)

print final

However, when I run this, I still see the keys that are the same:

[{'poker': {'John': {'Wins': 6, 'Losses': 1}, 'Jack': {'Wins': 6, 'Losses': 5}, 'Bill': {'Wins': 3, 'Losses': 0}}}, {'Blackjack': {'Bill': {'Wins': 4, 'Losses': 3}, 'John': {'Wins': 7, 'Losses': 0}, 'Jack': {'Wins': 1, 'Losses': 3}}}]

And what I expect to see:

[{"poker":{"John":{"Wins": 6}, "Jack": {"Losses": 5}, "Bill": {"Wins": 3, "Losses": 0}}}, {"Blackjack": {"Jack":{"Wins": 1, "Losses": 3}}}]

It would be nice if I could see the number differences between the old and new, but I'll just settle to only show what changed.

score 2 · Accepted Answer · answered Apr 09 '18 at 18:25

2

I think there's just a small bug in your code, if you fix it, things will work as you expected

The line

type(new(key)) == dict

always fails as you should access the element of a dictionary with new[key]. You can replace the line if inside try with

if isinstance(new[key], dict)

By the way, it's never a good idea to make this kind of silent try except.

answered Apr 09 '18 at 18:25

aliciawyy

151
3

2

There should be -50 rep penalty for anyone posting python code on SO which includes a bare except. – ekhumoro Apr 09 '18 at 18:33
Unfortunately in Python 2.x it is possible for third-party code to raise exceptions that do not inherit from Exception, so in Python 2.x there are some cases where you may have to use a bare except. @ekhumoro – Tyler Cowan Apr 09 '18 at 18:37
That did exactly what I was looking for, thanks! And excuse my noobish, self taught code. – DorklyDad Apr 09 '18 at 18:43
@TylerCowan. I think you are missing the point that there is no excuse for posting questions (like the current one) that can be easily resolved by simply reading a traceback. Of course, if there is no traceback, that is a different problem altogether (and obviously not relevant here). – ekhumoro Apr 09 '18 at 18:46
And apparently I missed that I used ('s instead of ['s. if type(new[key]) == dict: works too. Thanks for the help! I'll make sure to leave the silent fails off before another one of these. – DorklyDad Apr 09 '18 at 18:52
@DorklyDad yes type(new[key]) == dict also works, but I think isinstance is more pythonic. You're welcome! – aliciawyy Apr 09 '18 at 18:55
@ekhumoro I was simply outlining the minor caveats in both your and aliciawyy statements. It could be read out of context. Additionally, at least now you have explicitly said "there is no excuse for posting questions (like the current one) that can be easily resolved by simply reading a traceback". Which might not have been obvious to others. – Tyler Cowan Apr 09 '18 at 22:23

score 0 · Answer 2 · answered Apr 09 '18 at 18:16

You can use recursion with zip to get a full listing of changes:

old = [{"poker":{"John":{"Wins": 4, "Losses": 3}, "Jack": {"Wins": 6, "Losses": 1}}}, {"Blackjack":{"Bill": {"Wins": 4, "Losses": 3}, "John": {"Wins": 7, "Losses": 0}}}]
new = [{"poker":{"John":{"Wins": 6, "Losses": 3}, "Jack": {"Wins": 6, "Losses": 5}, "Bill": {"Wins": 3, "Losses": 0}}}, {"Blackjack":{"Bill": {"Wins": 4, "Losses": 3}, "John": {"Wins": 7, "Losses": 0}, "Jack": {"Wins": 1, "Losses": 3}}}]

def compare(player1, player2):
  return {a:compare(b, d) if isinstance(b, dict) and isinstance(d, dict) else {'old':min([b, d]), 'new':max([b, d])} for [a, b], [c, d] in zip(player1.items(), player2.items())}

final_data = [compare(p1, p2) for p1, p2 in zip(old, new)]

Output:

[{'poker': {'John': {'Wins': {'new': 6, 'old': 4}, 'Losses': {'new': 3, 'old': 3}}, 'Jack': {'Wins': {'new': 6, 'old': 6}, 'Losses': {'new': 5, 'old': 1}}}}, {'Blackjack': {'Bill': {'Wins': {'new': 4, 'old': 4}, 'Losses': {'new': 3, 'old': 3}}, 'John': {'Wins': {'new': 7, 'old': 7}, 'Losses': {'new': 0, 'old': 0}}}}]

The result adds one additional dictionary to the structure: a comparison between the original result and the updated data.

While it does add the comparison that I wanted, it also includes data that is the same, and with my nested structures, it's about 33k lines of data that I need to compare each time... so that's why I wanted to eliminate the exact matches. — DorklyDad, Apr 09 '18 at 18:43

python nested dictionary comparison

2 Answers2