Removing duplicate from list using list comprehension lead to all Nones

Question

I tried the following code to remove duplicates from list using list comprehension

lists = [1, 2, 2, 3, 4, 3, 5, 6, 1]

unique_lists = []
[unique_lists.append(x) for x in lists if x not in unique_lists]
print(unique_lists)

unique_lists = []
g = [unique_lists.append(x) for x in lists if x not in unique_lists]
print(g)

The printed result are shown as below

[1, 2, 3, 4, 5, 6]
[None, None, None, None, None, None]

I'm trying to understand why the 2nd method (simply by assigning the list to g, returns all None? Thanks.

`append()` doesn't return anything. Just do the list comprehension without assigning the result to something. `unique_lists` will still get filled. — Random Davis, Sep 14 '21 at 18:07

score 0 · Accepted Answer · answered Sep 14 '21 at 18:08

0

It's because list.append() always returns None. The easy way to do what you want is to convert it to a set.

unique_lists = list(set(lists))

answered Sep 14 '21 at 18:08

Tim Roberts

48,973
4
21
30

This will not preserve order. – Jab Sep 14 '21 at 18:19
1

Thanks. That makes sense. I should have thought about that. – Sean Sep 14 '21 at 19:29

Jab · Answer 2 · 2021-09-14T18:18:08.827

list.append returns None. What you could do is use a set to remember the unique numbers:

lists = [1, 2, 2, 3, 4, 3, 5, 6, 1]

uniques = set()
unique_lists = [x for x in lists if not (x in uniques or uniques.add(x))]

Or a more efficient way would be to use how dict's act similar to a set while keeping insertion order (in python3+):

unique_lists = list(dict.fromkeys(lists))

You can also check this question for more answers/explanation on how to remove dups from a list.

score 0 · Answer 3 · answered Sep 14 '21 at 19:15

Using a set to select the 1st occurrence of each distinct element will be more efficient than a list. You can even embed this control set in the comprehension:

g = [x for s in [set()] for x in lists if x not in s and not s.add(x)]

if you're not concerned with performance, you could select the item based on the index of their first occurrence:

g = [x for i,x in enumerate(lists) if i==lists.index(x)]

There is also a trick you can do using a dictionary:

g = list(dict(zip(lists,lists)))

Removing duplicate from list using list comprehension lead to all Nones

3 Answers3