18

I have a list of sets :

L = [set([1, 4]), set([1, 4]), set([1, 2]), set([1, 2]), set([2, 4]), set([2, 4]), set([5, 6]), set([5, 6]), set([3, 6]), set([3, 6]), set([3, 5]), set([3, 5])]

(actually in my case a conversion of a list of reciprocal tuples)

and I want to remove duplicates to get :

L = [set([1, 4]), set([1, 2]), set([2, 4]), set([5, 6]), set([3, 6]), set([3, 5])]

But if I try :

>>> list(set(L))
TypeError: unhashable type: 'set'

Or

>>> list(np.unique(L))
TypeError: cannot compare sets using cmp()

How do I get a list of sets with distinct sets?

Nathan
  • 8,093
  • 8
  • 50
  • 76
Covich
  • 2,544
  • 4
  • 26
  • 37
  • One way could be to convert the list of `set` to `list` of `list` and then remove the duplicates and then converting the `list` elements back to `set`. – ZdaR Aug 30 '15 at 13:18
  • @ZdaR I'm going to say this in bold: **Your hint is wrong.** Following the part of your instructions which say "*convert the list of `set` to `list` of `list` and then remove the dupicates*", we get that `list(map(list, [{3, 11}, {11, 3}]))` outputs `[[3, 11], [11, 3]]`. – wlad Mar 25 '17 at 13:15

4 Answers4

24

The best way is to convert your sets to frozensets (which are hashable) and then use set to get only the unique sets, like this

>>> list(set(frozenset(item) for item in L))
[frozenset({2, 4}),
 frozenset({3, 6}),
 frozenset({1, 2}),
 frozenset({5, 6}),
 frozenset({1, 4}),
 frozenset({3, 5})]

If you want them as sets, then you can convert them back to sets like this

>>> [set(item) for item in set(frozenset(item) for item in L)]
[{2, 4}, {3, 6}, {1, 2}, {5, 6}, {1, 4}, {3, 5}]

If you want the order also to be maintained, while removing the duplicates, then you can use collections.OrderedDict, like this

>>> from collections import OrderedDict
>>> [set(i) for i in OrderedDict.fromkeys(frozenset(item) for item in L)]
[{1, 4}, {1, 2}, {2, 4}, {5, 6}, {3, 6}, {3, 5}]
thefourtheye
  • 233,700
  • 52
  • 457
  • 497
  • @PM2Ring It exists in 2.7 so that statement is not strictly true. – Dan D. Aug 30 '15 at 14:05
  • @DanD: Oops! I originally just looked at the [Python 3 docs](https://docs.python.org/3/library/collections.html#collections.OrderedDict), where it says "New in version 3.1". But I just now checked in the Python 2 docs & it says "New in version 2.7."; I should've checked there to see if it had been back-ported. Sorry about that. I'll remove my comment(s). – PM 2Ring Aug 30 '15 at 14:34
4

An alternative using a loop:

result = list()
for item in L:
    if item not in result:
        result.append(item)
Paulo Almeida
  • 7,803
  • 28
  • 36
  • 1
    It is better to use `[]` than `list()` to create a list – Bhargav Rao Aug 30 '15 at 13:33
  • Why is that Bhargav Rao? – Reblochon Masque Aug 30 '15 at 14:13
  • 1
    @ReblochonMasque: it's because `[]` is literal syntax (allowing the empty list to be instantiated at compile time) whereas `list()` is a function call (the function name must be looked up at run time and then called to return the empty list). The latter is slightly more expensive. – Alex Riley Aug 30 '15 at 16:51
  • 2
    My preference for `list()` is merely aesthetic. Up until today I didn't even know it was slightly more expensive. If that's the only issue, I don't think it's that big of a deal. – Paulo Almeida Aug 30 '15 at 17:34
  • I could not find verifiable info, but it looks like operator overload to me; in that case, [] will call list() in the background... for most cases, the difference, if any, will be in the nanosecond scale! :) – Reblochon Masque Aug 31 '15 at 01:55
  • Other thing is you can shadow the list. So its better to prefer literals – thefourtheye Aug 31 '15 at 05:49
  • 1
    @ReblochonMasque I investigated a little and [they behave differently](http://stackoverflow.com/questions/2736693/creating-a-list-in-python-something-sneaky-going-on), but it's an insignificant difference, as you say. – Paulo Almeida Aug 31 '15 at 09:21
  • 1
    @thefourtheye I think that's a bit of a stretch, unless I'm missing something. You'd have to make the mistake of assigning to `list` instead of assigning `list()` to something. – Paulo Almeida Aug 31 '15 at 09:25
1

Here is another alternative

yourNewSet = map(set,list(set(map(tuple,yourSet))))
The6thSense
  • 8,103
  • 8
  • 31
  • 65
DaveQ
  • 1,642
  • 10
  • 15
  • **This answer is wrong**. Two equal sets can be mapped to two different tuples. I have seen this happen. For example: `ss = [{3, 11}, {11, 3}]; list(map(tuple, ss))` outputs `[(3, 11), (11, 3)]` – wlad Mar 25 '17 at 13:00
0

There is another alternative.

import itertools
list_sets = [set(['a', 'e', 'f']), set(['c', 'b', 'f']), set(['a', 'e', 'f']), set(['a', 'd']), set(['a', 'e', 'f'])]

lists = [list(s) for s in list_sets] # convert a list of sets to a list of lists
lists.sort()
lists_remove_duplicates = [lists for lists,_ in itertools.groupby(lists)]
print(lists_remove_duplicates)

# output
[['a', 'd'], ['a', 'e', 'f'], ['c', 'b', 'f']]
SparkAndShine
  • 17,001
  • 22
  • 90
  • 134