How to update values in a nested dictionary?

Question

I have 2 dictionaries:

data = {
  "filter":
    {
      "and":
        [
          {
            "or":
              [
                {
                  "and":
                    [
                      {"category": "profile", "key": "languages", "operator": "IN", "value": "EN"},
                      {"category": "skill", "key": "26366", "value": 100, "operator": "EQ"},
                    ],
                },
              ],
          },
          {"or": [{"category": "skill", "key": "45165", "operator": "NE"}]},
          {"or": [{"category": "skill", "key": "48834", "value": 80, "operator": "GT"}]},
          {"or": [{"category": "profile", "key": "gender", "operator": "EQ", "value": "FEMALE"}]},
        ],
    },
}

new_val = {'26366': '11616', '45165': '11613', '48834': '11618'}

I want to update values in "data" dictionary with the values from "new_val" dictionary.

So that 26366(in "data" dict) becomes 11616(from "new_val" dict), 45165 becomes 11613, and 48834 becomes 11618. "data" dictionary nesting can be different (both up and down)

The key in the "data" dictionary can be different, not only "key", it can be "skill_id", "filter_id" and so on.

And get this result:

{
  "filter":
    {
      "and":
        [
          {
            "or":
              [
                {
                  "and":
                    [
                      {"category": "profile", "key": "languages", "operator": "IN", "value": "EN"},
                      {"category": "skill", "key": "11616", "value": 100, "operator": "EQ"},
                    ],
                },
              ],
          },
          {"or": [{"category": "skill", "key": "11613", "operator": "NE"}]},
          {"or": [{"category": "skill", "key": "11618", "value": 80, "operator": "GT"}]},
          {"or": [{"category": "profile", "key": "gender", "operator": "EQ", "value": "FEMALE"}]},
        ],
    },
}

(1) Do you want **all** occurrences of `26366` to be replaced with `11616`, or only one? (2) Do you want to modify the dict in-place, or return an updated dict without modifying the old one? — Stef, Sep 19 '22 at 15:22
Can you get inspiration from this related question? [Printout specific keys and values in nested .json data recursively in python](https://stackoverflow.com/questions/68893096/printout-specific-keys-and-values-in-nested-json-data-recursively-in-python/68893319#68893319) — Stef, Sep 19 '22 at 15:27
@Stef yep, if there are several such values then update them all. Return a copy — Brze, Sep 19 '22 at 15:27
Or you can write a function `f` and use `map_in_depth` from this answer: [Dynamically accessing an element in a nested list python](https://stackoverflow.com/a/68996758/3080723). Edit: Ooops, sorry, wrong link, that's for a list of lists. — Stef, Sep 19 '22 at 15:29
@Stef pydantic has deep update method but it doesn't take into lists that may come across on the path. I try to expand this method(deep_update) to update dictionary with such nesting but without result — Brze, Sep 19 '22 at 15:34
Yes I'm afraid there is no way around the `isinstance(..., dict)` and `isinstance(...,list)` method for this. — Stef, Sep 19 '22 at 15:46
Welcome to Stack Overflow. *Where exactly do you get stuck* when attempting to write the code? For example, are you able to do any other kind of processing on the nested data? Are you able to do this kind of replacement on a simple dict? — Karl Knechtel, Sep 19 '22 at 15:51

Stef · Accepted Answer · 2022-09-19T15:49:39.297

To return an updated dict without modifying the old one:

def updated_in_depth(d, replace):
    if isinstance(d, dict):
        return {k: updated_in_depth(v, replace)
                for k,v in d.items()}
    elif isinstance(d, list):
        return [updated_in_depth(x, replace) for x in d]
    else:
        return replace.get(d, d)

Testing with your data and new_val:

>>> updated_in_depth(data, new_val)
{'filter': {'and': [{'or': [{'and': [
                            {'category': 'profile', 'key': 'languages', 'operator': 'IN', 'value': 'EN'},
                            {'category': 'skill', 'key': '11616', 'value': 100, 'operator': 'EQ'}]}]},
                    {'or': [{'category': 'skill', 'key': '11613', 'operator': 'NE'}]},
                    {'or': [{'category': 'skill', 'key': '11618', 'value': 80, 'operator': 'GT'}]},
                    {'or': [{'category': 'profile', 'key': 'gender', 'operator': 'EQ', 'value': 'FEMALE'}]}]}}

score 1 · Answer 2 · answered Sep 19 '22 at 15:26

1

Use something like this:

data['filter']['and']['or']['and'][1]['key']='11616'

answered Sep 19 '22 at 15:26

Yep, but I'm trying to make a function, since there can be a lot of values – Brze Sep 19 '22 at 15:37

score 1 · Answer 3 · answered Sep 19 '22 at 15:32

To search for the keys recursively you can do:

from copy import deepcopy


def replace(d, new_vals):
    if isinstance(d, dict):
        # replace key (if there's match):
        if "key" in d:
            d["key"] = new_vals.get(d["key"], d["key"])
        for v in d.values():
            replace(v, new_vals)
    elif isinstance(d, list):
        for v in d:
            replace(v, new_vals)


new_data = deepcopy(data)
replace(new_data, new_val)
print(new_data)

Prints:

{
    "filter": {
        "and": [
            {
                "or": [
                    {
                        "and": [
                            {
                                "category": "profile",
                                "key": "languages",
                                "operator": "IN",
                                "value": "EN",
                            },
                            {
                                "category": "skill",
                                "key": "11616",
                                "value": 100,
                                "operator": "EQ",
                            },
                        ]
                    }
                ]
            },
            {"or": [{"category": "skill", "key": "11613", "operator": "NE"}]},
            {
                "or": [
                    {
                        "category": "skill",
                        "key": "11618",
                        "value": 80,
                        "operator": "GT",
                    }
                ]
            },
            {
                "or": [
                    {
                        "category": "profile",
                        "key": "gender",
                        "operator": "EQ",
                        "value": "FEMALE",
                    }
                ]
            },
        ]
    }
}

If you don't need copy of data you can omit the deepcopy:

replace(data, new_val)
print(data)

score 1 · Answer 4 · answered Sep 19 '22 at 15:35

1

You can build a recursive function like this

def walk_dict(d):
    if isinstance(d, list):
        for item in d:
            walk_dict(item)
    elif isinstance(d, dict):
        if 'key' in d and d['key'] in new_val:
            d['key'] = new_val[d['key']]
        for k, v in d.items():
            walk_dict(v)


walk_dict(data)
print(data)

answered Sep 19 '22 at 15:35

Ghilas BELHADJ

13,412
10
59
99

This appears to be missing a second argument for `new_val` – Stef Sep 19 '22 at 15:47

score 0 · Answer 5 · answered Sep 19 '22 at 15:47

As many have advised, a recursive function will do the trick:

def a(d):
    if isinstance(d, dict): # if dictionary, apply a to all values
        d = {k: a(d[k]) for k in d.keys()}
        return d
    elif isinstance(d, list): # if list, apply to all elements
        return [a(x) for x in d]
    else: # apply to d directly (it is a number, a string or a bool)
        return new_val[d] if d in new_val else d

When a is called, it check what is the type of the variable d:

if d is a list, it apply a to each element of the list and return the updated list
if d is a dict, it applies a to all values and return the updated dict
otherwise, it returns the mapped new value if the old one has been found in the new_val keys

score 0 · Answer 6 · answered Sep 19 '22 at 16:01

data = {
  "filter":
    {
      "and":
        [
          {
            "or":
              [
                {
                  "and":
                    [
                      {"category": "profile", "key": "languages", "operator": "IN", "value": "EN"},
                       {"category": "skill", "key": "11616", "value": 100, "operator": "EQ"},
                    ],
                },
              ],
          },
          {"or": [{"category": "skill", "key": "11613", "operator": "NE"}]},
          {"or": [{"category": "skill", "key": "11618", "value": 80, "operator": "GT"}]},
          {"or": [{"category": "profile", "key": "gender", "operator": "EQ", "value": "FEMALE"}]},
        ],
        },
}
class Replace:
    def __init__(self,data):
        self.data=data
    def start(self,d):
        data = self.data
    
        
        def replace(data):
            if type(data) == list:
                for v in data:
                    replace(v)
            if type(data) == dict:
                for k,v in data.items():
                    if type(v) == dict:
                        replace(v)
                    if type(v) == str:
                        if v in d:
                            data[k] = d[v]
    
        replace(data)
        return data
new_data = Replace(data).start({'26366': '11616',
                                '45165': '11613',
                                '48834': '11618'})
print(new_data)

How to update values in a nested dictionary?

6 Answers6

Linked