How to check if the dictionaries in a list are unique?

Question

I have found some approaches, but unfortunately none, where it comes to the contents of Dictioranies. I hope this question is not a duplicate.

My Code:

import os
import csv
import glob

os.chdir("/Users/x/PycharmProjects/x")

listme = list()

for csvs in glob.glob("*.csv"):
    with open(csvs, mode="r") as csvfile:
        csv_reader = csv.DictReader(csvfile)

        for row in csv_reader:
            spek = {"name": row['name'], "party": row['group'], "term":row['term']}
            listme.append(spek)

for element in listme:
    print(element)

My Output (about 5000 elements):

{'name': 'Erwin Stahl', 'party': 'SPD', 'term': '9'}
{'name': 'Eugen Glombig', 'party': 'SPD', 'term': '9'}
{'name': 'Eugen von der Wiesche', 'party': 'SPD', 'term': '9'}
{'name': 'Ferdinand Tillmann', 'party': 'CDU', 'term': '9'}
{'name': 'Franz Amrehn', 'party': 'CDU', 'term': '9'}
{'name': 'Franz Handlos', 'party': 'CSU', 'term': '9'}
{'name': 'Franz Heinrich Krey', 'party': 'CDU', 'term': '9'}
{'name': 'Franz Josef Conrad', 'party': 'CDU', 'term': '9'}

How can I now check if in a term, a person of the same name exits from the same party only once? The name and party combination may occur, but not in the same term.

Thank you in advance!

Knowing how to compare dictionaries might help: https://stackoverflow.com/questions/1911273/is-there-a-better-way-to-compare-dictionary-values/5635309#5635309 — Ben Jones, Jul 17 '18 at 15:20

score 1 · Answer 1 · answered Jul 17 '18 at 15:22

You need to turn your dict objects into sth hashable. One option would be to turn their items into a frozenset:

s = set()
for d in listme:
  frz = frozenset(d.items())  # hashable data structure
  if frz in s:
    print(d)  # print duplicates
  else:
    s.add(frz)

score 0 · Accepted Answer · answered Jul 17 '18 at 15:23

0

My attempt (using Counter from itertools):

data = [{'name': 'Erwin Stahl', 'party': 'SPD', 'term': '9'},
{'name': 'Eugen Glombig', 'party': 'SPD', 'term': '9'},
{'name': 'Eugen von der Wiesche', 'party': 'SPD', 'term': '9'},
{'name': 'Ferdinand Tillmann', 'party': 'CDU', 'term': '9'},
{'name': 'Franz Amrehn', 'party': 'CDU', 'term': '9'},
{'name': 'Franz Handlos', 'party': 'CSU', 'term': '9'},
{'name': 'Franz Heinrich Krey', 'party': 'CDU', 'term': '9'},
{'name': 'Franz Josef Conrad', 'party': 'CDU', 'term': '9'}]

from collections import Counter

# Find duplicates based on 'name', 'party', 'term' - you can edit which keys you want to search for:
c = Counter((d['name'], d['party'], d['term']) for d in data)

new_data = []
for d in data:
    if c[(d['name'], d['party'], d['term'])] == 1:
        new_data.append(d)

print(new_data)

answered Jul 17 '18 at 15:23

Andrej Kesely

168,389
15
48
91

what happends in your solution, if a dublicate is founded? – madik_atma Jul 17 '18 at 15:42
@madik_atma If duplicate is found then it is not appended to new_data list. – Andrej Kesely Jul 17 '18 at 15:44
thats mean, if I add a else and print there if != 1, they should print the dublicate row? – madik_atma Jul 17 '18 at 15:46
@madik_atma Yes, it depends on you what you want to do. In variable c you have number of duplicates (1 means no duplicates). – Andrej Kesely Jul 17 '18 at 15:48
I want only catch possible dublicates, if theire exists. Because, I dont know it now :) – madik_atma Jul 17 '18 at 15:50

How to check if the dictionaries in a list are unique?

2 Answers2