Removing duplicate strings from a list in python

Question

If I have a string list ,

a = ["asd","def","ase","dfg","asd","def","dfg"]

how can I remove the duplicates from the list?

score 101 · Accepted Answer · answered Nov 20 '11 at 08:32

Convert to a set:

a = set(a)

Or optionally back to a list:

a = list(set(a))

Note that this doesn't preserve order. If you want to preserve order:

seen = set()
result = []
for item in a:
    if item not in seen:
        seen.add(item)
        result.append(item)

See it working online: ideone

score 15 · Answer 2 · answered Nov 20 '11 at 08:32

15

Use the set type to remove duplicates

a = list(set(a))

answered Nov 20 '11 at 08:32

Krumelur

31,081
7
77
119

murasing · Answer 3 · 2014-02-07T09:59:49.943

6

def getUniqueItems(iterable):
result = []
for item in iterable:
    if item not in result:
        result.append(item)
return result

print (''.join(getUniqueItems(list('apple'))))

P.S. Same thing like one of the answers here but a little change, set is not really required !

edited Feb 07 '14 at 09:59

answered Feb 07 '14 at 09:49

murasing

412
6
15

does this code snippet preserve the order? – tumultous_rooster Jun 09 '14 at 06:16
It does preserve order, but has quadratic complexity, and that is bad for any list remotely big. – ReneSac Mar 29 '15 at 03:35
this awesome! thanks – raditya gumay Sep 24 '16 at 14:57

score 5 · Answer 4 · answered Nov 20 '11 at 08:33

5

You could pop 'em into a set and then back into a list:

a = [ ... ]
s = set(a)
a2 = list(s)

answered Nov 20 '11 at 08:33

rossipedia

56,800
10
90
93

Removing duplicate strings from a list in python

4 Answers4

Linked