one liner to go through an iterable (generator)

Question

I encountered some code looking like:

[func(val) for val in iterable]

There is an iterable (in my case a generator) for which a user wants to call a function for each value for its side effects (func could for example just be print) but where the return value is irrelevant.

What I don't like about this approach is, that a temporary list is created, which might consume quite some memory if the generator yields a lot of values.

If the return value of func always evaluates to False, then following works:

any(func(val) for val in iterable)

If the return value of func always evaluates to True, then following works:

all(func(val) for val in iterable)

What would I have to do if the return value of func can evaluate to True or to False

Anything better then forcing the value to False?

The best I came up with is:

any(func(val) and False for val in iterable)

or

all(func(val) or True for val in iterable)

Could you maybe clarify or give an example of the expected output/result? If is maily the memory you are afraid of, just use `(func(val) for val in iterable)` which is a constructor that only yields the result when asked, thus all the answers are not held in memory when initiated. — Jakob Guldberg Aaes, Jun 16 '20 at 12:28
In fact the person did not want to write a two line: `for val in iterable():` followed by `func(val)` He wanted to write an elegant and fast performing way of calling a function for each member of an iterable and used a list expression. SO her looking for a one liner (ideally) an expression, that calls a funciton for each member of an iterable, but doesn't bild a list for nothing — gelonida, Jun 16 '20 at 12:30
not sure if this is true (too lazy to uncompile and compare), but I have the impression, that a `for` statement is slightly less performant than using `any` or `all`, but I'm not 100% sure about this. I encountered above code and was curious about the best way to improve it (mem and performance wise) — gelonida, Jun 16 '20 at 12:34

LeopardShark · Accepted Answer · 2020-06-16T12:08:30.870

4

Probably just

for val in iterable:
   func(val)

is clearest.

for val in iterable: func(val)

is available if a one-liner is truly necessary.

edited Jun 16 '20 at 12:08

answered Jun 16 '20 at 12:07

LeopardShark

3,820
2
19
33

1

agreed, but that's a two liner. I wanted to suggest another one liner to the person who wrote the one liner with a list. – gelonida Jun 16 '20 at 12:08
1

Yep, just after commenting I remembered , that `for` can be a one liner – gelonida Jun 16 '20 at 12:09
1

You could hide that `for` loop in a function somewhere. I.e., `def exhaustIterable(iterable): ...` – Daniel Walker Jun 16 '20 at 12:09
1

for as one liner is OK. I was more wondering whether there is already something builtin, that behaves like your suggested `exhaustIterable` – gelonida Jun 16 '20 at 12:10
1

will probably accept this answer. only minor draw back is, that it cannot be used as a `lambda`, as it is not an expression, but a statement, But this was not part of my question. – gelonida Jun 16 '20 at 12:14

ywbaek · Answer 2 · 2020-06-17T01:34:22.067

1

How about using a set with bool function?

{bool(func(val)) for val in iterable}

EDITED:
After looking at @gelonida's analysis, I believe the following is a bit faster.

{None for val in iterable(N) if func(val)}

edited Jun 17 '20 at 01:34

answered Jun 16 '20 at 13:33

ywbaek

2,971
3
9
28

1

Yep second solution is better. It avoids determining the hash of `func`'s return values. Your second solution gave me an alternative idea: `[None for val in iterable(N) if func(val) and False]` but is not more performant than your solution and not more readable – gelonida Jun 17 '20 at 01:52

gelonida · Answer 3 · 2020-06-17T02:14:02.567

Just a synthesis and timing analysis of the given answers / potential solutions

It seems that @LeopardShark's answer is the shortest, most readable answer. and amongst the fastest. (timeit is not exact and I didn't look at the byte codes)

Speed wise - @LeopardShark's answer - @ywbaek's second suggestion - The initial code, that I found if N is not too big (~10000) - the suggestions, that I posted in my question

The initial code has the drawback of allocating releasing memory for nothing.

The code I suggested in my question has the drawback to be less intuitive to understand and if messing up the combination of (all, any) and (and False, or True) one might not execute everything as expected and is also a little less performant

@ywbaek's solution is safer than my suggestions and about as intuitive, but is executing a little faster.

The simplest solution has the minor drawback of not being usable as lambda.

My code for timing:

N=10000
M=500

called = 0
def func(v):
    global called
    called += 1
    v * v * v * v * v *v / (v+0.1)

def iterable(N):
    for v in range(N):
        v * 2
        yield v

def testrun():
    global called
    called=0
    print(timeit(test, number=M), end=" ")
    print(called)

print("consume some CPU")
timeit(lambda: 3**.5 **.2) # it seems, that timeit is a little more predictable if I let the process warm up a little

print("Start measures")

def test():
    for val in iterable(N): func(val)
testrun()

def test():
    {None for val in iterable(N) if func(val)}
testrun()

def test():
    [func(val) for val in iterable(N)]
testrun()

def test():
    all(func(val) or True for val in iterable(N))
testrun()

def test():
    any(func(val) and False for val in iterable(N))
testrun()

results on my old PC:

consume some CPU
Start measures
3.864932143012993 5000000
3.916696268017404 5000000
4.0817033689818345 5000000
4.293206526956055 5000000
4.319622751965653 5000000

one liner to go through an iterable (generator)

3 Answers3