searching a list for duplicate values in python 2.7

Question

list1 = ["green","red","yellow","purple","Green","blue","blue"]

so I got a list, I want to loop through the list and see if the colour has not been mentioned more than once. If it has it doesn't get append to the new list.

so u should be left with

list2 = ["red","yellow","purple"]

So i've tried this

list1 = ["green","red","yellow","purple","Green","blue","blue","yellow"]
list2 =[]
num = 0
for i in list1:
    if (list1[num]).lower == (list1[i]).lower:
        num +=1
    else:
        list2.append(i)
        num +=1

but i keep getting an error

Answer 1

Use a Counter: https://docs.python.org/2/library/collections.html#collections.Counter.

from collections import Counter
list1 = ["green","red","yellow","purple","green","blue","blue","yellow"]
list1_counter = Counter([x.lower() for x in list1])
list2 = [x for x in list1 if list1_counter[x.lower()] == 1]

Note that your example is wrong because yellow is present twice.

Answer 2

The first problem is that for i in list1: iterates over the elements in the list, not indices, so with i you have an element in your hands.

Next there is num which is an index, but you seem to increment it rather the wrong way.

I would suggest that you use the following code:

for i in range(len(list1)):
    unique = True
    for j in range(len(list1)):
        if i != j and list1[i].lower() == list1[j].lower():
            unique = False
            break
    if unique:
        list2.append(list1[i])

How does this work: here i and j are indices, you iterate over the list and with i you iterate over the indices of element you potentially want to add, now you do a test: you check if somewhere in the list you see another element that is equal. If so, you set unique to False and do it for the next element, otherwise you add.

You can also use a for-else construct like @YevhenKuzmovych says:

for i in range(len(list1)):
    for j in range(len(list1)):
        if i != j and list1[i].lower() == list1[j].lower():
            break
    else:
        list2.append(list1[i])

You can make the code more elegant by using an any:

for i in range(len(list1)):
    if not any(i != j and list1[i].lower() == list1[j].lower() for j in range(len(list1))):
        list2.append(list1[i])

Now this is more elegant but still not very efficient. For more efficiency, you can use a Counter:

from collections import Counter

ctr = Counter(x.lower() for x in list1)

Once you have constructed the counter, you look up the amount of times you have seen the element and if it is less than 2, you add it to the list:

from collections import Counter

ctr = Counter(x.lower() for x in list1)

for element in list1:
    if ctr[element.lower()] < 2:
        list2.append(element)

Finally you can now even use list comprehension to make it very elegantly:

from collections import Counter

ctr = Counter(x.lower() for x in list1)

list2 = [element for element in list1 if ctr[element.lower()] < 2]

Answer 3

Here's another solution:

list2 = []
for i, element in enumerate(list1):
    if element.lower() not in [e.lower() for e in list1[:i] + list1[i + 1:]]:
        list2.append(element)

Answer 4

By combining the built-ins, you can keep only unique items and preserve order of appearance, with just a single empty class definition and a one-liner:

from future_builtins import map  # Only on Python 2 to get generator based map
from collections import Counter, OrderedDict

class OrderedCounter(Counter, OrderedDict):
    pass

list1 = ["green","red","yellow","purple","Green","blue","blue"]

# On Python 2, use .iteritems() to avoid temporary list
list2 = [x for x, cnt in OrderedCounter(map(str.lower, list1)).items() if cnt == 1]

# Result: ['red', 'yellow', 'purple']

You use the Counter feature to count an iterable, while inheriting from OrderedDict preserves key order. All you have to do is filter the results to check and strip the counts, reducing your code complexity significantly.

It also reduces the work to one pass of the original list, with a second pass that does work proportional to the number of unique items in the list, rather than having to perform two passes of the original list (important if duplicates are common and the list is huge).

Answer 5

Here's yet another solution...

Firstly, before I get started, I think there is a typo... You have "yellow" listed twice and you specified you'd have yellow in the end. So, to that, I will provide two scripts, one that allows the duplicates and one that doesn't.

Original:

list1 = ["green","red","yellow","purple","Green","blue","blue","yellow"]
list2 =[]
num = 0
for i in list1:
    if (list1[num]).lower == (list1[i]).lower:
        num +=1
    else:
        list2.append(i)
        num +=1

Modified (does allow duplicates):

colorlist_1 = ["green","red","yellow","purple","Green","blue","blue","yellow"]
colorlist_2 = []
seek = set()

i = 0
numberOfColors = len(colorlist_1)

while i < numberOfColors:
    if numberOfColors[i].lower() not in seek:
        seek.add(numberOfColors[i].lower())
        colorlist_2.append(numberOfColors[i].lower())

    i+=1

print(colorlist_2)

# prints ["green","red","yellow","purple","blue"]

Modified (does not allow duplicates):

EDIT

Willem Van Onsem's answer is totally applicable and already thorough.