3

For example, if I had:

a = ["apples", "bananas", "cucumbers", "bananas"]

How could I remove the duplicate "bananas" so that:

a = ["apples", "bananas", "cucumbers"]

Also, if I had:

a = ["apples", "bananas", "cucumbers"]

b = ["pears", "apples", "watermelons"]

How could I remove the duplicate "apples" from both lists so that:

a = ["bananas", "cucumbers"]

b = ["pears", "watermelons"]
macabeus
  • 4,156
  • 5
  • 37
  • 66
HashtagAbuse
  • 49
  • 4

5 Answers5

5

The set-based solutions don't retain the order of the items. The following will keep the items in order and delete all but the first occurrence of each, using an auxilary set to keep track of which items have already been seen.

seen = set()
a = [seen.add(item) or item for item in a if item not in seen]

If you want to reuse the same list object, you can do that this way:

seen = set()
a[:] = (seen.add(item) or item for item in a if item not in seen)
kindall
  • 178,883
  • 35
  • 278
  • 309
3

Use built-in functions set

a = ["apples", "bananas", "cucumbers", "bananas"]
a = list(set(a))
print(a)

In second case, use list comprehension

a = ["apples", "bananas", "cucumbers"]
b = ["pears", "apples", "watermelons"]

r = [i for i in a if i not in b] + [i for i in b if i not in a]    
print(r)
macabeus
  • 4,156
  • 5
  • 37
  • 66
2

The key to doing this is using Python's set.

  • In Python, a set is a data structure in which every item is unique.
  • If you call set(list), with a list as a parameter, you will get a set that contains all of the elements in list, with the duplicates removed
  • You can then convert this back into a list by calling list().

So, in your first example, you can write 

a = list(set(a))

There are a couple of other methods in set that are useful.

  1. Intersection - Calling set1.intersection(set2) returns a set with all of the objects that are in both set1 and set2.
  2. Difference - Calling set1.difference(set2) returns a set with all of the elements in set1 that are not in set2.

So, in your second example, you can write

set1 = set(a).intersection(set(b)) #Get elements that are in both lists
set2 = set(a).difference(set1) #Get a set elements that are in a but not in b
a = list(set2) #Convert back to a list
Erik Godard
  • 5,930
  • 6
  • 30
  • 33
1

You can just use set():

a = ["apples", "bananas", "cucumbers", "bananas"]

print list(set(a))
Simon
  • 9,762
  • 15
  • 62
  • 119
0

You can use a set object to record the duplicate elements. Like this:

def handle_dumplicate(*lsts):
    s = set()
    result = []
    for lst in lsts:
        no_dump_lst = []
        for ele in lst:
            if ele in s:
                continue
            s.add(ele)
            no_dump_lst.append(ele)
        result.append(no_dump_lst)
    return result

a = ["apples", "bananas", "cucumbers"]
b = ["pears", "apples", "watermelons"]

a, b = handle_dumplicate(a, b)
print a
print b
SamChi
  • 59
  • 4