Python list comprehension: list sub-items without duplicates

Question

I am trying to print all the letters in all the words in a list, without duplicates. I tried:

>>> wordlist = ['cat','dog','rabbit']
>>> letterlist = []
>>> [[letterlist.append(x) for x in y] for y in wordlist]
[[None, None, None], [None, None, None], [None, None, None, None, None, None]]
>>> letterlist
['c', 'a', 't', 'd', 'o', 'g', 'r', 'a', 'b', 'b', 'i', 't']

The desired result is ['c', 'a', 't', 'd', 'o', 'g', 'r', 'b', 'i'] instead.

How do I modify the list comprehension to remove duplicates?

You can do `[letterlist.append(x) for y in wordlist for x in y if x not in letterlist]` — karthikr, Jul 21 '13 at 16:29

score 10 · Answer 1 · answered Jul 21 '13 at 16:28

10

Do you care about maintaining order?

>>> wordlist = ['cat','dog','rabbit']
>>> set(''.join(wordlist))
{'o', 'i', 'g', 'd', 'c', 'b', 'a', 't', 'r'}

answered Jul 21 '13 at 16:28

roippi

25,533
4
48
73

4

This can be written as `set().union(*wordlist)` which allows `set` to take care of the multiple iterables and doesn't require joining them to a string first – Jon Clements Jul 21 '13 at 16:53

score 4 · Answer 2 · answered Jul 21 '13 at 16:59

Two approaches:

Preserving order:

>>> from itertools import chain
>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(chain.from_iterable(wordlist)))
['c', 'a', 't', 'd', 'o', 'g', 'r', 'b', 'i']

If you're not fussed about order:

>>> list(set().union(*wordlist))
['a', 'c', 'b', 'd', 'g', 'i', 'o', 'r', 't']

Neither of this are using list-comps for side effects, eg:

[[letterlist.append(x) for x in y] for y in wordlist]

Is building a list of lists of Nones purely to mutate letterlist

score 3 · Answer 3 · answered Jul 21 '13 at 16:31

3

While all other answers don't maintain order, this code does:

from collections import OrderedDict
letterlist = list(OrderedDict.fromkeys(letterlist))

See also, an article about several ways with benchmarks: Fastest way to uniqify a list in Python.

answered Jul 21 '13 at 16:31

Moayad Mardini

7,271
5
41
58

This answer doesn't seem to work: `>>> wordlist = list(OrderedDict.fromkeys(wordlist)) >>> wordlist ['cat', 'dog', 'rabbit']` – Mingyu Jul 21 '13 at 16:57
Please note that my code uses 'letterlist' rather than 'wordlist' which your code uses. – Moayad Mardini Jul 21 '13 at 21:45

score 2 · Accepted Answer · edited Aug 01 '16 at 17:12

2

If you want to edit you own code:

[[letterlist.append(x) for x in y if x not in letterlist] for y in wordlist]

or

list(set([[letterlist.append(x) for x in y if x not in letterlist] for y in wordlist]))

else:

list(set(''.join(wordlist)))

edited Aug 01 '16 at 17:12

Peter Badida

11,310
10
44
90

answered Jul 21 '13 at 16:27

rnbguy

1,369
1
10
28

score 0 · Answer 5 · answered Jul 21 '13 at 16:27

0

You can use set to remove the duplicates but the order is not maintained.

>>> letterlist = list({x for y in wordlist for x in y})
>>> letterlist
['a', 'c', 'b', 'd', 'g', 'i', 'o', 'r', 't']
>>>

answered Jul 21 '13 at 16:27

zhangyangyu

8,520
2
33
43

score 0 · Answer 6 · answered Jul 21 '13 at 16:28

0

wordlist = ['cat','dog','rabbit']
s = set()
[[s.add(x) for x in y] for y in wordlist]

answered Jul 21 '13 at 16:28

Udy

2,492
4
23
33

Python list comprehension: list sub-items without duplicates

6 Answers6

Linked

Related