Search for duplicates inside an array

Question

I am trying to find duplicates inside an array. my array looks like:

['LM_Auto', 'LM_Auto', 'LM_Op', 'LM_Op']

and much longer with a few thousand of these pairs.

def clear_stemmed_LM(collection): 
i = 0
ele = i+1
for i in collection:
    for ele in collection:
        if i == ele:
            del collection[ele]
return collection

the error says the list indices must be integers or slices not strings. How can I compare strings in an array?

[if x+1 for in x range] #Idea

or something similar like this?

Do you need to find the position of the duplicates for each unique occurrence? Or do you just want to remove the duplicates? Is the order of appearance important? — norok2, Jan 17 '20 at 10:35
I just want to remove duplicates. The order doesn't matter, I just need a cleared list which contains each string only once — Yannisch, Jan 17 '20 at 10:39

score 1 · Answer 1 · answered Jan 17 '20 at 10:41

1

If you just need to purge the duplicates in the list collection, as it seems from the code, list(set(collection)).

This is a very, very common and basic issue. I encourage you to Google thoroughly first!

answered Jan 17 '20 at 10:41

Lester Jack

179
8

score 0 · Answer 2 · answered Jan 17 '20 at 10:34

0

for i in collection iters over the items so in i you'll find the first string not the index. Use for i in range(len(collection)) instead

answered Jan 17 '20 at 10:34

User

806
1
11
28

Jesus, it happens too often to me. Thank you! – Yannisch Jan 17 '20 at 10:42

Search for duplicates inside an array

2 Answers2