Find duplicate list in a list of lists?

Question

I have a list of lists as so:

data = [[4,5,0],[3,4,0],[4,5,0],[1,2,0],[1,2,0],[4,5,0],[3,4,0],[2,3,0]]

What I want is to return a list of unique lists (order doesn't matter) as:

unique_data =[[4,5,0],[3,4,0],[1,2,0],[2,3,0]]

My first thought was to do:

for x,y in zip(data, data[1:])):
        if x == y:
               unique_data.append(x)

But this only compares two lists that are directly next to each other. Any other ideas are greatly appreciated. Thank you.

Answer 1

One thing you could do is turn the lists into tuples (since tuples are hashable and can be put in a set), then convert that to a set to filter out duplicates, and back again.

data = [[4,5,0],[3,4,0],[4,5,0],[1,2,0],[1,2,0],[4,5,0],[3,4,0],[2,3,0]]

unique_data = [list(tup) for tup in set(tuple(lst) for lst in data)]
# [[2, 3, 0], [3, 4, 0], [4, 5, 0], [1, 2, 0]]

Answer 2

Iterate over your data, and save it only if it isn't already in the save list

data = [[4, 5, 0], [3, 4, 0], [4, 5, 0], [1, 2, 0], [1, 2, 0], [4, 5, 0], [3, 4, 0], [2, 3, 0]]

result = []
for x in data:
    if x not in result:
        result.append(x)

Answer 3

You can try this way using a set to remove duplicates.

Since lists cannot be hashed,

• Convert the individual lists to tuples.
• Add them to a set so that only unique tuples will be present inside it.
• Convert the tuples back to lists as you need list of lists.

data = [[4,5,0],[3,4,0],[4,5,0],[1,2,0],[1,2,0],[4,5,0],[3,4,0],[2,3,0]]
data = [tuple(x) for x in data]
ans = [list(x) for x in list(set(data))]
print(ans)

[[2, 3, 0], [3, 4, 0], [4, 5, 0], [1, 2, 0]]