I have a list of lists and want to remove duplicates from it. I have a way to work but it looks ugly and I'm not sure if there's a better way to do it.
a_list = [[1,2],[2,3],[1,2]]
print [list(tup) for tup in set([tuple(x) for x in a_list])]
which prints:
[[1,2],[2,3]]
which is what I want. I'm using small lists for this, so am not concerned with speed but am interested in if there is a nicer way to do this.
You can make it a little shorter:
print list(set(tuple(x) for x in a_list))
Output:
[(1, 2), (2, 3)]
If you're not so worried about the display format, then simply leave off the outermost list conversion, giving:
set([(1, 2), (2, 3)])
I ran a timer on the three answers given so far, using /usr/shar/dict/words as the second part of each tuple, a simple 1 for the first element. The file has nearly half a million (479,829) words.
0.517719984055 list->set->list (your original code) 1.30243611336 list->set->map (tmajest's suggestion) ??? build new list (Salman's suggestion)
The last one has not finished in over 6 minutes of run time. top does show that it still has full use of one CPU.
>>> a_list=[[1,2],[2,3],[1,2]]
>>> newlist=[]
>>> for each in a_list:
if each not in newlist:
newlist.append(each)
>>> newlist
[[1, 2], [2, 3]]
Make a new empty list. If any list from a_list is not present in newlist, append it to newlist. Hope this helps. :)
If the end result must be lists, then there's not really a shorter way. You can use map instead of list comprehensions if you want:
list(map(list, set(map(tuple, a_list))))
