In python, how to check if a list of lists has duplicates?

Question

I searched for the answer to my question and found that many said to try set(my_list), but in my case (I have lists within a list) python throws an error:

Traceback (most recent call last):
  File "E:/Users/Rene/PycharmProjects/vaip/Main.py", line 17, in <module>
  set(punktid)
TypeError: unhashable type: 'list'

So my question is, how can I find and remove duplicates in a list like this:

my_list = [[0, 3], [3, 4], [5, 6], [9, 2], [0, 3]]

This question might help to understand the whole hashing thing: [Hashing question](http://stackoverflow.com/questions/2671376) — Kyle_S-C, Nov 16 '14 at 14:38

vaultah · Answer 1 · 2014-11-16T14:50:34.600

3

Converting all sublists to tuples makes it possible to create set (all elements become hashable):

>>> {tuple(x) for x in my_list}
{(5, 6), (9, 2), (0, 3), (3, 4)}

In this case we use braces to denote a set comprehension, though it's also possible to use ordinary set constructor:

>>> set(tuple(x) for x in my_list)
{(5, 6), (9, 2), (0, 3), (3, 4)}

If your final result must be a list of lists (the order may not be preserved):

>>> [list(y) for y in {tuple(x) for x in my_list}]
[[5, 6], [9, 2], [0, 3], [3, 4]]

edited Nov 16 '14 at 14:50

answered Nov 16 '14 at 14:09

vaultah

44,105
12
114
143

Might be worth pointing out that `{...}` instantiates the stuff inside as a set. – Kyle_S-C Nov 16 '14 at 14:40

score 0 · Answer 2 · answered Nov 16 '14 at 14:41

You can use,

>>> l
[[0, 3], [3, 4], [5, 6], [9, 2], [0, 3]]
>>> def remove_duplicate(l):
    tup = [tuple(x) for x in l]
    set_tup = set(tup)
    return [list(x) for x in list(set_tup) ]

>>> print remove_duplicate(l)
[[5, 6], [9, 2], [0, 3], [3, 4]]

In python, how to check if a list of lists has duplicates?

2 Answers2