Remove duplicate items in an array with not same-ordered inside elements

Question

Our example list:

(
    [(), (2, 0)], 
    [(2,), (0,)], 
    [(), (0, 2)], 
    [(0,), (2,)]
)

I want to be able to remove duplicates in the list, and by this I mean same inside elements.

This means the elements [(2,), (0,)] and [(0,), (2,)] in the list are same. So in summary, i want to remove different order inside elements. Note that this example is of 2 inside elements but I want it for any number. I want to keep whatever comes first as long as they are not both (or a many duplicates we have) there.

I thought about sorting the inner elements, converting to str them checking for duplicates, but I am not sure if this is the way and I don't know how to make it

For example an element of [(),(2,0),(1,)] is the same as [(),(1,),(2,0)].

Is your list a list of `str` types, or of `list` types? Not sure if a typo by you, or intentional. — felipe, Mar 17 '20 at 20:37
I edited the list to remove the str of elements. The outer one is list — Solving Py, Mar 17 '20 at 20:40
The fact that you have 3 levels of list nesting makes this a bit trickier. Is `[(), (0,2), (1,)]` the same as `[(), (2,0),(1,)]`? — Owen, Mar 17 '20 at 20:56
it is important the order of elements(lists) after removing the duplicates? — kederrac, Mar 17 '20 at 21:00
Does this answer your question? [Python: Remove Sublists from List if Same Including/Not Including Order](https://stackoverflow.com/questions/20462664/python-remove-sublists-from-list-if-same-including-not-including-order) — mkrieger1, Mar 17 '20 at 21:10

score 2 · Accepted Answer · answered Mar 17 '20 at 21:05

2

you could use fozensets:

l = (
    [(), (2, 0)], 
    [(2,), (0,)], 
    [(), (0, 2)], 
    [(0,), (2,)]
)


r = set()

result = tuple()
for e in l:
    f = frozenset(e)
    if f not in r:
        result += (e,)
        r.add(f)
result

output:

([(), (2, 0)], [(2,), (0,)], [(), (0, 2)]

if the order of the lists is not important you can use:

tuple(list(e) for e in {frozenset(e) for e in l})

output:

([(), (0, 2)], [(2,), (0,)], [(2, 0), ()])

answered Mar 17 '20 at 21:05

kederrac

16,819
6
32
55

@Todd no, the elements from the tuples inside of the lists are not allowed to change the order, only the tuples inside lists are allowed to change the order – kederrac Mar 17 '20 at 21:20
1

You do the same using `sorted(e)` and `result` as a list. But with `frozenset` it's little bit faster. Good answer by the way. – Chiheb Nexus Mar 17 '20 at 21:45

score 0 · Answer 2 · answered Mar 17 '20 at 21:00

One way to solve this, is to sort the values in the list and remove check if the nth element exists in the next part of the list

from itertools import compress

lst = (
    [(), (2, 0)], 
    [(2,), (0,)], 
    [(), (0, 2)], 
    [(0,), (2,)]
)

_ = list(map(lambda x: x.sort(), lst))
list(compress(lst, [x not in lst[i+1:] for i, x in enumerate(lst)]))

Remove duplicate items in an array with not same-ordered inside elements

2 Answers2