0

How could I remove duplicates two dimension list without changing the order?

list_j = [[100,2,3,3], [4,98,99,98], [5,99,98,4], [5,99,98,5], [100,99,98,100,100,6]]

list_pre = [list(set(i)) for i in list_j]
print(list_pre)

[[2, 3, 100], [98, 99, 4], [98, 99, 4, 5], [98, 99, 5], [98, 99, 100, 6]]

As you can see it changed the order. What I want is [[100,2,3,],...]

Desired output [[100,2,3,], [4,98,99], [5,99,98,4], [5,99,98], [100,99,98,6]]

Ella
  • 361
  • 3
  • 9
  • 1
    Does this answer your question? [how can I maintain sequence of my list using set?](https://stackoverflow.com/questions/3562971/how-can-i-maintain-sequence-of-my-list-using-set) – quamrana Jan 09 '21 at 16:16

3 Answers3

5

Use OrderedDict to maintain the order of insertion of keys:

from collections import OrderedDict
list_j = [[100,2,3,3], [4,98,99,98], [5,99,98,4], [5,99,98,5], [100,99,98,100,100,6]]
output = [list(OrderedDict.fromkeys(x)) for x in list_j]

gives

[[100, 2, 3], [4, 98, 99], [5, 99, 98, 4], [5, 99, 98], [100, 99, 98, 6]]

If you using Python 3.7 or higher, you can use normal dictionaries as well since they also maintain the order of insertion:

output = [list(dict.fromkeys(x)) for x in list_j]
Jarvis
  • 8,494
  • 3
  • 27
  • 58
  • @Javis - Spot on. Acutally, it depends on the Python version (>3.5)? You can get all `dict() items` guaranteed in the insert order. So it could be: [list(dict.fromkeys(ll)) for ll in list_j] – Daniel Hao Jan 09 '21 at 16:28
0

Same question here:

def f7(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq if not (x in seen or seen_add(x))]

list_j = [[100,2,3,3], [4,98,99,98], [5,99,98,4], [5,99,98,5], [100,99,98,100,100,6]]

list_pre = [f7(i) for i in list_j]

print(list_pre)

This will give the desired output

mpountou
  • 798
  • 8
  • 15
0

Try this:

def remove_duplicates(my_list):
    result_list = []
    for _list in my_list:
        result_list.append(sorted(set(_list), key=lambda ind: _list.index(ind)))
    return result_list
Higs
  • 384
  • 2
  • 7