python RuntimeError: dictionary changed size during iteration

Question

I have obj like this

{hello: 'world', "foo.0.bar": v1, "foo.0.name": v2, "foo.1.bar": v3}

It should be expand to

{ hello: 'world', foo: [{'bar': v1, 'name': v2}, {bar: v3}]}

I wrote code below, splite by '.', remove old key, append new key if contains '.', but it said RuntimeError: dictionary changed size during iteration

def expand(obj):
    for k in obj.keys():
        expandField(obj, k, v)

def expandField(obj, f, v):
    parts = f.split('.')
    if(len(parts) == 1):
        return
    del obj[f]
    for i in xrange(0, len(parts) - 1):
        f = parts[i]
        currobj = obj.get(f)
        if (currobj == None):
            nextf = parts[i + 1]
            currobj = obj[f] = re.match(r'\d+', nextf) and [] or {}
        obj = currobj
    obj[len(parts) - 1] = v

for k, v in obj.iteritems():

RuntimeError: dictionary changed size during iteration

**Please search before you post** - http://stackoverflow.com/search?q=RuntimeError%3A+dictionary+changed+size+during+iteration — D.Shawley, Apr 11 '12 at 14:16
possible duplicate of [How to fix this python error? RuntimeError: dictionary changed size during iteration](http://stackoverflow.com/questions/2844837/how-to-fix-this-python-error-runtimeerror-dictionary-changed-size-during-itera) — Felix Kling, Apr 11 '12 at 14:16
@D.Shawley Ironically this question is now the first hit on Google when searching for the error message. — kasperd, Nov 21 '14 at 03:23

score 30 · Accepted Answer · answered Apr 11 '12 at 14:15

30

Like the message says: you changed the number of entries in obj inside of expandField() while in the middle of looping over this entries in expand.

You might try instead creating a new dictionary of the form you wish, or somehow recording the changes you want to make, and then making them AFTER the loop is done.

answered Apr 11 '12 at 14:15

Scott Hunter

48,888
12
60
101

4

Yep, a new dictionary is the way to go. This also simplifies the recursion. – Niklas B. Apr 11 '12 at 14:15

Damien · Answer 2 · 2022-09-12T18:14:53.160

16

You might want to copy your keys in a list and iterate over your dict using the latter, eg:

def expand(obj):
    keys = list(obj.keys())  # freeze keys iterator into a list
    for k in keys:
        expandField(obj, k, v)

I let you analyse if the resulting behavior suits your expected results.

Edited as per comments, thank you !

edited Sep 12 '22 at 18:14

answered Feb 11 '14 at 22:07

Damien

1,624
2
19
26

4

might work better with`keys = list(obj.keys())` which copies the keys into a list – Kai Carver Apr 23 '20 at 04:35
`keys = obj.keys()` works nicely. No need to add the memory overhead of converting to a list. – S3DEV Oct 27 '20 at 09:10
4

`keys = list(obj.keys())` is necessary if you delete any key of the dict while iterating over it (no matter if you iterate over the dict itself or using its keys). This is because `obj.keys()` returns just a view on the dict and thus reflects all changes done to the dict. Using the `list(...)` constructor will actually copy the keys to a new, independent list. – Splines Feb 23 '21 at 09:55

score 3 · Answer 3 · answered Sep 14 '13 at 21:24

I had a similar issue with wanting to change the dictionary's structure (remove/add) dicts within other dicts.

For my situation I created a deepcopy of the dict. With a deepcopy of my dict, I was able to iterate through and remove keys as needed.Deepcopy - PythonDoc

A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

Hope this helps!

score 2 · Answer 4 · answered Jun 14 '20 at 13:35

For those experiencing

RuntimeError: dictionary changed size during iteration

also make sure you're not iterating through a defaultdict when trying to access a non-existent key! I caught myself doing that inside the for loop, which caused the defaultdict to create a default value for this key, causing the aforementioned error.

The solution is to convert your defaultdict to dict before looping through it, i.e.

d = defaultdict(int)
d_new = dict(d)

or make sure you're not adding/removing any keys while iterating through it.

been scratching my head over this! – Ashish Ranjan Aug 07 '22 at 06:43 — Ashish Ranjan, Aug 07 '22 at 06:43

score 0 · Answer 5 · answered May 07 '20 at 02:19

0

Rewriting this part

def expand(obj):
    for k in obj.keys():
        expandField(obj, k, v)

to the following

def expand(obj):
    keys = obj.keys()
    for k in keys:
        if k in obj:
            expandField(obj, k, v)

shall make it work.

answered May 07 '20 at 02:19

mausamsion

230
1
8

python RuntimeError: dictionary changed size during iteration

5 Answers5

Linked

Related