How to remove k,v entries from a list of dictionaries

Question

I want to remove the values I don't need from an existing json file:

{ "items": [ { "id": "abcd", "h": 2, "x": 0, "level": 4 }, { "id": "dfgg", "h": 7, "x": 5, "level": 30 } ] }

I've tried to remove the values in place but get 'dictionary changed size during iteration'.

with open('inventory2.json', 'r') as inf:
    data = json.load(inf)
    inf.close()

    keysiwant = ['x', 'h']
    for dic in data['items']:
        for k, v in dic.items():
            if k not in keysiwant:
                dic.pop(k, None)

Possible duplicate of [How to avoid "RuntimeError: dictionary changed size during iteration" error?](https://stackoverflow.com/questions/11941817/how-to-avoid-runtimeerror-dictionary-changed-size-during-iteration-error) — Patrick Artner, Jan 21 '19 at 19:31

Patrick Artner · Accepted Answer · 2019-01-21T19:39:38.620

Problem: dict.items() in python 3 is just a view - not a copy of the dict's items - you cannot change it while iterating.

You can however put the dict.items() iterator into a list() (which copies it and decouples it from the dict that way) - you can then iterate over the copy of the dict.items() instead:

import json

t = """{ "items": [ { "id": "abcd", "h": 2, "x": 0, "level": 4 }, 
                    { "id": "dfgg", "h": 7, "x": 5, "level": 30 } ] }"""

data = json.loads(t)   # loads is better for SO-examples .. it makes it a mcve
keysiwant = ['x', 'h']
for dic in data['items']:
    for k, v in list(dic.items()):
        if k not in keysiwant:
            dic.pop(k, None)

print(data)

Output:

{'items': [{'h': 2, 'x': 0}, {'h': 7, 'x': 5}]}

More on python2/python3 dict.items(): in this answer to What is the difference between dict.items() and dict.iteritems()?

json.loads() gave an error, still used json.load(), otherwise, nailed it thank you :) — Scott, Jan 21 '19 at 19:39
@Scott `json.loads()` is to load json from a _`string`_ not a file ;) hence better for demonstration as [mcve] on SO — Patrick Artner, Jan 21 '19 at 19:40

score 0 · Answer 2 · answered Jan 21 '19 at 21:21

Please try this. It uses fewer iterations because it filters out keys first, before sending them to pop/remove. Also, it uses keys only (list(dic)) instead of tuple key/value.

import json

t = """{ "items": [ { "id": "abcd", "h": 2, "x": 0, "level": 4 },
                    { "id": "dfgg", "h": 7, "x": 5, "level": 30 } ] }"""

data = json.loads(t)
keysiwant = ["x", "h"]

for dic in data["items"]:
    for k in (k for k in list(dic) if k not in keysiwant):
        dic.pop(k, None)

print(data)

Output:

{'items': [{'h': 2, 'x': 0}, {'h': 7, 'x': 5}]}

Works just as well maybe slightly faster on the few comparisons I ran. Thank you! — Scott, Jan 22 '19 at 23:20

How to remove k,v entries from a list of dictionaries

2 Answers2