How do I remove dicts from a list with duplicate fields in python?

Question

Let's say I have a list of dicts. I define "duplicates" as any two dicts in the list that have the same value for the field "id" (even if the other fields are different). How do I remove these duplicates.

An example list would be something like:

[{'name': 'John' , 'id':1}, {'name': 'Mike' , 'id':5},{'name': 'Dan' , 'id':5}]

In this case, 'Mike' and 'Dan' would be duplicates, and one of them needs to be removed. It doesn't matter which one.

This is a duplicate of a question from yesterday http://stackoverflow.com/questions/11092511/python-list-of-unique-dictionaries — John La Rooy, Jun 20 '12 at 07:01
It's a little different, since that user wanted to remove dicts that were completely identical, whereas I wanted to remove dicts that were identical in a single field, even if all other fields were different. — , Jun 20 '12 at 07:45
No, the OP clarified in a comment that this is exactly what they wanted to do. The solution is identical. Even the field name is the same — John La Rooy, Jun 21 '12 at 00:45

score 10 · Accepted Answer · answered Jun 20 '12 at 07:00

10

Dump them into another dictionary, then pull them out after.

dict((x['id'], x) for x in L).values()

answered Jun 20 '12 at 07:00

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

score 2 · Answer 2 · edited Jun 20 '12 at 18:21

2

The following function's list comprehension should solve your problem.

def f(seq):
    s = set()
    return [x for x in seq if x['id'] not in s and not s.add(x['id'])]

edited Jun 20 '12 at 18:21

octopusgrabbus

10,555
15
68
131

answered Jun 20 '12 at 07:44

dbykov

31
3

How do I remove dicts from a list with duplicate fields in python?

2 Answers2

Linked