Check unique values for a key in a list of dicts

Question

I have a list of dictionaries where I want to drop any dictionaries that repeat their id key. What's the best way to do this e.g:

example dict:

product_1={ 'id': 1234, 'price': 234}

List_of_products[product1:, product2,...........]

How can I the list of products so I have non repeating products based on their product['id']

"dictionaries that repeat their id key" I don't understand. What are you trying to do? — , May 31 '16 at 13:59
Instead of a list you could use a dict which maps id to product_dict. — syntonym, May 31 '16 at 14:02
@LutzHorn OP has multiple dictionaries with a key `id` that probably occur in several of the dictionaries and their values happen to be the same. He or She would want to weed out those some how. Aka loop over and remove duplicate `id` values from dictionaries in a list structure. — Torxed, May 31 '16 at 14:03

score 3 · Answer 1 · edited May 23 '17 at 12:24

Select one of product dictionaries in which the values with the same id are different. Use itertools.groupby,

import itertools

list_products= [{'id': 12, 'price': 234},
                {'id': 34, 'price': 456},   
                {'id': 12, 'price': 456},
                {'id': 34, 'price': 78}]


list_dicts = list()
for name, group in itertools.groupby(sorted(list_products, key=lambda d : d['id']), key=lambda d : d['id']):
    list_dicts.append(next(group))

print(list_dicts)
# Output
[{'price': 234, 'id': 12}, {'price': 456, 'id': 34}]

If the product dictionaries with the same id are totally the same, there is an easier way as described in Remove duplicate dict in list in Python. Here is a MWE.

list_products= [{'id': 12, 'price': 234},
                {'id': 34, 'price': 456},   
                {'id': 12, 'price': 234},
                {'id': 34, 'price': 456}]               

result = [dict(t) for t in set([tuple(d.items()) for d in list_products])]

print(result)
# Output
[{'price': 456, 'id': 34}, {'price': 234, 'id': 12}]

score 2 · Accepted Answer · answered May 31 '16 at 14:10

2

a = [{'id': 124, 'price': 234}, {'id': 125, 'price': 234},  {'id': 1234, 'price': 234},  {'id': 1234, 'price': 234}]
a.sort()
for indx, val in enumerate(a):
    if val['id'] == a[indx+1]['id']:
        del a[indx]

answered May 31 '16 at 14:10

binu.py

1,137
2
9
20

perfect nice and simple thanks – Yunti May 31 '16 at 14:42
It might occur `IndexError: list index out of range`, for instance, `a = [{'id': 124, 'price': 234}, {'id': 125, 'price': 234}, {'id': 34, 'price': 234}]` – SparkAndShine May 31 '16 at 14:45
@sparkandshine no it wont because enumerate takes care of that – binu.py May 31 '16 at 15:31
Hmm I may have spoken to soon, I get a TypeError: unorderable types: dict() < dict(). Doesn't sort() just work on the list index? How can it sort the list based on one of the dictionary keys within the list? – Yunti Jun 08 '16 at 19:59
@Yunti looks like you are trying to sort on dictionary, which is not a part of this question. can you elaborate more? – binu.py Jun 09 '16 at 06:48
Based on your answer I had to use this to get it to work.` for index, dic in enumerate(self.dic_id_list): if dic['dicId'] in [dic['dicId'] for dic in self.dic_id_list[index + 1:]]: del self.dic_id_list[index]` – Yunti Jun 09 '16 at 18:16
This error is different. the dic here for index, dic in enumerate(self.dic_id_list): and the dic here: for dic in self.dic_id_list[index + 1:]]: are different but as you have given same name they are overriding and creating issue. – binu.py Jun 10 '16 at 05:59
Upvoting because contrary to the linked duplicate, this is readable code and does not rely on one liners which are incomprehensible without extensive language knowledge. – Kenn Sebesta Oct 11 '18 at 14:46

Check unique values for a key in a list of dicts

2 Answers2