Remove dictionaries from a list of dictionaries under certain conditions

Question

I have a very large list of dictionaries that looks like this (I show a simplified version):

list_of_dicts:
[{'ID': 1234,
  'Name': 'Bobby',
  'Animal': 'Dog',
  'About': [{'ID': 5678, 'Food': 'Dog Food'}]},
 {'ID': 5678, 'Food': 'Dog Food'},
 {'ID': 91011,
  'Name': 'Jack',
  'Animal': 'Bird',
  'About': [{'ID': 1996, 'Food': 'Seeds'}]},
 {'ID': 1996, 'Food': 'Seeds'},
 {'ID': 2007,
  'Name': 'Bean',
  'Animal': 'Cat',
  'About': [{'ID': 2008, 'Food': 'Fish'}]},
 {'ID': 2008, 'Food': 'Fish'}]

I'd like to remove the dictionaries containing IDs that are equal to the ID's nested in the 'About' entries. For example, 'ID' 2008, is already nested in the nested 'About' value, therefore I'd like to remove that dictionary.

I have some code that can do this, and for this specific example it works. However, the amount of data that I have is much larger, and the remove() function does not seem to remove all the entries unless I run it a couple of times.

Any suggestions on how I can do this better?

My code:

nested_ids = [5678, 1996, 2008]
for i in list_of_dicts:
    if i['ID'] in nested_ids:
        list_of_dicts.remove(i)

Desired output:

[{'ID': 1234,
  'Name': 'Bobby',
  'Animal': 'Dog',
  'About': [{'ID': 5678, 'Food': 'Dog Food'}]},
 {'ID': 91011,
  'Name': 'Jack',
  'Animal': 'Bird',
  'About': [{'ID': 1996, 'Food': 'Seeds'}]},
 {'ID': 2007,
  'Name': 'Bean',
  'Animal': 'Cat',
  'About': [{'ID': 2008, 'Food': 'Fish'}]}]

Does this answer your question? [How to remove items from a list while iterating?](https://stackoverflow.com/questions/1207406/how-to-remove-items-from-a-list-while-iterating) — ILS, Nov 01 '22 at 14:04

score 3 · Accepted Answer · answered Nov 01 '22 at 13:56

3

You can use a list comprehension:

cleaned_list = [d for d in list_of_dicts if d['ID'] not in nested_ids]

answered Nov 01 '22 at 13:56

jprebys

2,469
1
11
16

score 2 · Answer 2 · answered Nov 01 '22 at 13:55

It is happening because we are modifying the dict while iterating it, So to avoid that we can copy the required values to a new dict as follow

filtered_dicts = []
nested_ids = [5678, 1996, 2008]
for curr in list_of_dicts:
    if curr['ID'] not in nested_ids:
        filtered_dicts.append(curr)

score 1 · Answer 3 · answered Nov 01 '22 at 13:54

1

the problem is that when you remove a member of a list you're changing the indexes of everything after that index so you should reorder the indices you want to remove in reverse so you start from the back of the list

so all you need to do is iterate over the list in reverse order:

for i in list_of_dicts[::-1]:

answered Nov 01 '22 at 13:54

vencaslac

2,727
1
18
29

1

Actually, `list_of_dicts[:]` is ok. – ILS Nov 01 '22 at 14:00

Remove dictionaries from a list of dictionaries under certain conditions

3 Answers3