4

From what I know, we may use generator when we want to use the values only once. Using the 2 examples below, my logic is that the 2nd one should be faster, because the first one creates a list first and then loop over the values.. while the 2nd only process the values from the generator. Yet, when I calculate the time, the list comprehension is always faster then the generator. Why is this?

1st:

x = []
a = time.perf_counter()
for j in [i**2 for i in range(20000)]:
    x.append(j)
print( time.perf_counter() - a )

2nd:

x = []
a = time.perf_counter()
for j in (i**2 for i in range(20000)):
    x.append(j)
print( time.perf_counter() - a )
Arief
  • 199
  • 1
  • 12
  • 1
    Which python version are you using? Because range returned a list in python 2 and is returning a generator in python 3 ... – quant Nov 16 '18 at 11:28
  • @quant Python 3, but both examples also contain `range` – Arief Nov 16 '18 at 11:34
  • Can you post the code you used to time the 2 approaches? – Ma0 Nov 16 '18 at 11:43
  • @Ev.Kounis have added the time calculation. – Arief Nov 16 '18 at 11:48
  • Possible duplicate of [List comprehension vs generator expression's weird timeit results?](https://stackoverflow.com/questions/11964130/list-comprehension-vs-generator-expressions-weird-timeit-results) – jpp Nov 16 '18 at 12:10

1 Answers1

6

Yeah, generators and genexprs are generally (heh) slower than list comprehensions, but on the other hand they're lazily evaluated, and you don't have to pay the memory cost for a fully precomputed list either. I imagine the speed difference is caused by call frame overhead with the (implicit or explicit) yield throwing values around.

Using your code, but timeit to measure it, and with a third version using a generator function:

import timeit

def f1():
    x = []
    for j in [i**2 for i in range(20000)]:
        x.append(j)
    return x

def f2():
    x = []
    for j in (i**2 for i in range(20000)):
        x.append(j)
    return x


def f3():
    def gen():
        for i in range(20000):
            yield i ** 2
    x = []
    for j in gen():
        x.append(j)
    return x


print(timeit.timeit(f1, number=100))
print(timeit.timeit(f2, number=100))
print(timeit.timeit(f3, number=100))

The results (Python 3.7.0) seem to point to genexprs being exactly as fast as generator functions, about 4-5% slower than the list comprehension.

f1 = 2.882695159
f2 = 3.0303254170000002
f3 = 3.002670741
AKX
  • 152,115
  • 15
  • 115
  • 172