list of dicts- get the number of duplications

Question

I have a list of dicts (same format) like this :

L =  [
        {'id': 1, 'name': 'john', 'age': 34},
        {'id': 1, 'name': 'john', 'age': 34},
        {'id': 2, 'name': 'hanna', 'age': 30},
        {'id': 2, 'name': 'hanna', 'age': 30},
        {'id': 3, 'name': 'stack', 'age': 40}
    ]

I want to remove duplication and get the number of this duplication like this

[
    {'id': 1, 'name': 'john', 'age': 34, 'duplication': 2},
    {'id': 2, 'name': 'hanna', 'age': 30, 'duplication': 2},
    {'id': 3, 'name': 'stack', 'age': 40, 'duplication': 1}
]

I already managed to remove the duplication by using a set.... but I can't get the number of duplications

my code :

no_duplication = [dict(s) for s in set(frozenset(d.items()) for d in L)]
            
no_duplication =  [
                   {'id': 1, 'name': 'john', 'age': 34},
                   {'id': 2, 'name': 'hanna', 'age': 30},
                   {'id': 3, 'name': 'stack', 'age': 40}
                  ]

number of dicts - number of not duplication shall give you number of duplication, no ? — DonKnacki, Apr 27 '21 at 12:19
Mathieu, we can't apply Counter to dicts! DonKnacki, can you please clarify I didn't get it — salah mazroai, Apr 27 '21 at 12:23
Does this answer your question? [Python: count repeated elements in the list](https://stackoverflow.com/questions/23240969/python-count-repeated-elements-in-the-list) — Tomerikoo, Apr 27 '21 at 12:27
`len(L) - len(no_duplication)` don't give you the expected result ? maybe I misunderstood — DonKnacki, Apr 27 '21 at 12:37

score 1 · Accepted Answer · answered Apr 27 '21 at 12:24

1

Here is a solution you can give a try using collections.Counter,

from collections import Counter

print([
    {**dict(k), "duplicated": v}
    for k, v in Counter(frozenset(i.items()) for i in L).items()
])

[{'age': 34, 'duplicated': 2, 'id': 1, 'name': 'john'},
 {'age': 30, 'duplicated': 2, 'id': 2, 'name': 'hanna'},
 {'age': 40, 'duplicated': 1, 'id': 3, 'name': 'stack'}]

answered Apr 27 '21 at 12:24

sushanth

8,275
3
17
28

ah I see using Counter instead of the set, I didn't think about it thank you – salah mazroai Apr 27 '21 at 12:29

score 0 · Answer 2 · answered Apr 27 '21 at 12:25

 ar = [
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 2, 'name': 'hanna', 'age': 30},
    {'id': 2, 'name': 'hanna', 'age': 30},
    {'id': 3, 'name': 'stack', 'age': 40}
]

br = []
cnt = []

for i in ar:
    if i not in br:
        br.append(i)
        cnt.append(1)
    else:
        cnt[br.index(i)] += 1

for i in range(len(br)):
    br[i]['duplication'] = cnt[i]

The desired output is contained in br as:

[
 {'id': 1, 'name': 'john', 'age': 34, 'duplication': 2}, 
 {'id': 2, 'name': 'hanna', 'age': 30, 'duplication': 2}, 
 {'id': 3, 'name': 'stack', 'age': 40, 'duplication': 1}
]

list of dicts- get the number of duplications

2 Answers2