Dynamic self-referencing conditional in list comprehension

Question

Goal: Create a conditional statement in a list comprehension that (1) dynamically tests -- i.e., upon each iteration -- if the element is not in the list being comprehended given (2) the list is itself updated on each iteration.

Background code:

arr = [2, 2, 4]
l = list()

Desired output:

l = [2, 4]

Desired behavior via for loop:

for element in arr:
        if element not in l:
            l.append(element)

Incorrect list comprehension not generating desired behavior:

l = [element for element in arr if element not in l]

Question restated: How do I fix the list comprehension so that it generates the desired behavior, i.e., the desired output stated above?

You cannot alter the same list you are making the comprehension into — Luis Masuelli, Mar 30 '16 at 15:27
Do you need to use a list comprehension for this? Is a set OK? — Mad Physicist, Mar 30 '16 at 15:27
If you simply want to remove duplicate you might want to have a look at http://stackoverflow.com/questions/480214/how-do-you-remove-duplicates-from-a-list-in-python-whilst-preserving-order — SwiftsNamesake, Mar 30 '16 at 15:27
@LuisMasuelli: This code isn't trying to; its trying to *access* the list it is building while it is being built, but with the same (lack of) success. — Scott Hunter, Mar 30 '16 at 15:28
The name `l` is not bound to the result of the comprehension until the comprehension runs to completion. — Mad Physicist, Mar 30 '16 at 15:29
Also, check out PEP 8 https://www.python.org/dev/peps/pep-0008/. Using a lower case L as a single variable name is strongly advised against. — Trev Davies, Mar 30 '16 at 15:30
Another alternative (if you really want to us a comprehension for whatever reason) is to use enumerate to keep track of the element's position within the original list, like so: `[e for i, e in enumerate(arr) if e not in arr[i+1:]]`. — SwiftsNamesake, Mar 30 '16 at 15:30
You denied what I said but said exactly the same, with few wording changes. As @MadPhysicist says, the name `l` is not yet bound. @SwiftsNamesake gave you a workaround. — Luis Masuelli, Mar 30 '16 at 15:31
You can't use a list comprehension because the list being generated doesn't have a named reference that you can use. You can create a list and use a generator to update it. `l = [];l.append(e for e in arr if e not in l)`. — tdelaney, Mar 30 '16 at 15:34
@tdelaney. Way ahead of you, but yes you can put it in a comprehension. — Mad Physicist, Mar 30 '16 at 15:34
@tdelaney. Also, you want `extend`, not `append` for that, but clever nevertheless. — Mad Physicist, Mar 30 '16 at 15:39
@MadPhysicist Oops... meant to do that. `l = [];l.extend(e for e in arr if e not in l)` — tdelaney, Mar 30 '16 at 15:40
@tdelaney. Do I have your permission to add that to my answer? — Mad Physicist, Mar 30 '16 at 15:41
@MadPhysicist I don't think encouraging people to follow a very common style guide is a bad thing. I find PyCharm's constant reminders regarding pep8 to be really useful in helping me present my code better. — Trev Davies, Mar 30 '16 at 15:43
@MadPhysicist - yeah, no problem. I only made it a comment because it wasn't a direct answer but I think it would be a good addition to yours. — tdelaney, Mar 30 '16 at 15:44
@Remuze. Please see the first section of PEP8 after the introduction. The title is pretty self-explanatory. Here is a link: https://www.python.org/dev/peps/pep-0008/#id13. Here is a link explaining the origin of the term bikeshedding: https://en.wikipedia.org/wiki/Law_of_triviality — Mad Physicist, Mar 30 '16 at 15:52
re. the bikeshedding and the lowercase l, I'll add that, as you discover the joys of pdb in debugging, you learn not to use 1 letter variable names that clash with pdb commands. — JL Peyret, Mar 30 '16 at 16:05

Mad Physicist · Answer 1 · 2016-03-30T15:45:59.373

If you absolutely must use a list comprehesion, you can just recast your for loop into one. The downside is that you will end up with a list of None elements, since that is what list.append returns:

>>> arr = [2, 2, 4]
>>> l = list()
>>> _ = [l.append(element) for element in arr if element not in l]
>>> print(l)
[2, 4]
>>> print(_)
[None, None]

If you are tied to comprehensions, but not necessarily to list comprehensions, you can use the generator comprehension suggested by @tdelaney. This will not create any unwanted byproducts and will do exactly what you want.

>>> arr = [2, 2, 4]
>>> l = list()
>>> l.extend(element for element in arr if element not in l)

A better way than either would probably be to put the original list into a set and then back into a list. The advantage of using a set to extending a list is that sets are much faster at adding elements after checking for prior containment. A list has to do a linear search and reallocate every time you add an element.

>>> l = list(set(arr))

score 0 · Answer 2 · answered Apr 04 '20 at 16:34

if you want to remove duplicates why not use set(the list containing duplicates) or list(dict.fromkeys(the list containing duplicates)?

but to answer your Question:

i think the whole thing is just wrong, l (your list) doesn't get updated with each iteration since it's inside the list comprehension

Dynamic self-referencing conditional in list comprehension

2 Answers2