0

This thought just came to my mind. Say for whatever reason you wanted to get the unique elements of a list via a list comprehension in Python.

[i if i in {created_comprehension} else 0 for i in [1, 2, 1, 2, 3]

[1, 2, 0, 0, 3]

I dunno, I don't really have a purpose for this but it'd be cool if it was possible to refer to the comprehension as it's being created.

(e.g. How to remove duplicate items from a list using list comprehension? is a similar question)

Community
  • 1
  • 1
aspin
  • 309
  • 2
  • 12
  • Unless there is some magic I am not aware of, you're better off using a for loop and updating another list in there. – squiguy Nov 12 '15 at 06:26

3 Answers3

5

I'll assume i in {created_comprehension} was meant to be i not in {created_comprehension}. At least that's what the data suggests.

So here's a fun horrible abuse that I wouldn't trust to always work. Mainly to demonstrate that the argument "it's impossible because it's not yet assigned" is wrong. While the list object indeed isn't assigned yet, it does already exist while it's being built.

>>> import gc
>>> [i if i not in self else 0
     for ids in [set(map(id, gc.get_objects()))]
     for self in [next(o for o in gc.get_objects() if o == [] and id(o) not in ids)]
     for i in [1, 2, 1, 2, 3]]
[1, 2, 0, 0, 3]

This gets the ids of all objects tracked for garbage collection before the new list gets created, and then after it got created, we find it by searching for a newly tracked empty list. Call it self and then you can use it. So the middle two lines are a general recipe. I also successfully used it for this question, but it got closed before I could post.

A nicer version:

>>> [i if i not in self else 0
     for old in [ids()] for self in [find(old)]
     for i in [1, 2, 1, 2, 3]]
[1, 2, 0, 0, 3]

That used these helper functions:

def ids():
    import gc
    return set(map(id, gc.get_objects()))

def find(old):
    import gc
    return next(o for o in gc.get_objects() if o == [] and id(o) not in old)
superb rain
  • 5,300
  • 2
  • 11
  • 25
  • 3
    Wow. This is... horrifying. In an amusing way, but still, horrifying. :-) – ShadowRanger Dec 10 '20 at 03:35
  • @ShadowRanger Added a nicer version now. I'd say it's almost ready for putting it on PyPI. – superb rain Dec 10 '20 at 04:17
  • Hah. (Silly) Suggestion: Incorporate `gc.disable()` at the beginning of `ids` followed by `gc.enable()` at the end of `find` so you prevent generation changes in between and can look solely at the youngest generation (calling `gc.get_objects(0)`) to reduce the number of objects examined. Clearly performance is the most important thing in this insanity. :-) (Not sure this is guaranteed to work; should check if a `gc` might occur between `BUILD_LIST` and `ids()` call; might help to wrap `ids()` as `(ids(),)` to avoid making a `list`). – ShadowRanger Dec 10 '20 at 04:27
  • @ShadowRanger Yeah that sounds great, given how the youngest generation is usually rather small. I also thought about optimizing space by only storing the ids of empty lists. I'm using 3.9, thought `in [ids()]` already doesn't build a list but [directly assigns](https://docs.python.org/3/whatsnew/3.9.html#optimizations). Turns out it's neither, it's `BUILD_TUPLE` instead (and the iteration over it). Odd. For `[find(old)]` it's `STORE_FAST` as expected. – superb rain Dec 10 '20 at 04:45
1

Disclaimer: this is purely speculation on my part, and I don't have data to back it up

I don't think you can refer to a list comprehension as it is being built. Python will first have to create the list, allocate memory or it, and add elements to it, before it binds it to a variable name. Therefore, I think you'll end up with a NameError if you try to refer to the list, while it's being built in a list-comp

You might ultimately, therefore, want a set to hold your uniques, and build your list from there (Oh God! this is hacky):

In [11]: L = [1, 2, 1, 2, 3]

In [12]: s = set(L)

In [13]: answer = [sub[0] for sub in [(i,s.remove(i)) if i in s else (0,0) for i in L]]

In [14]: answer
Out[14]: [1, 2, 0, 0, 3]

In [15]: s
Out[15]: set()
inspectorG4dget
  • 110,290
  • 27
  • 149
  • 241
  • Until CPython 2.6 it was possible to access the list using `locals()['_[1]']`: `[x if x not in locals()['_[1]'] else 0 for x in L]`. For this I find the good ol' *seen set* hack much easier to understand: `seen = set(); [x if x not in seen and not seen.add(x) else 0 for x in L]`. – Ashwini Chaudhary Nov 12 '15 at 06:51
0

Disclaimer: this is just an experiment. I compare a list comprehension and a list inside a list comprehension.

I want x to contain elements from [1,2,1,2,3,4,5] only if those elements are in this list comprehension [e for e in range(3,6)] which should be [3,4,5]

x = [i for a in [e for e in range(3,6)] for i in [1,2,1,2,3,4,5] if i == a]

The output is right:

[3, 4, 5]
Joe T. Boka
  • 6,554
  • 6
  • 29
  • 48