python basic level generator and list questions

Question

my_nums =(i*i for i in [1,2,3,4,5])
for k in (my_nums):
    print(k)
GG = list(my_nums)

print(GG)

It prints:

1
4
9
16
25
[]

Process finished with exit code 0

I can't understand why is [] value empty(it should be [1,4,9,16,25])? Also, does for-loop convert generator values to list?

Answer 1

Following is a generator comprehension:

my_nums =(i*i for i in [1,2,3,4,5])

for k in (my_nums):
    print(k)

So first time you loop over it and print the values, the generator prints every value it can generate. The for loop works by invoking my_nums.next() and assigning the value obtained to k, which then gets printed. The iteration of the for loop stops when the StopIteration exception is raised.

And, when you use GG = list(my_nums) after the for loop, you get an empty list since the generator is already exhausted and it has nothing to generate anymore.

If you want to store the values yielded by the generator into a list you can directly do that as following:

my_nums =(i*i for i in [1,2,3,4,5])
GG = list(my_nums) # no for loop for printing it already

Although, I am not sure that performance wise you would gain any advantages if you do it as above.

Answer 2

Simply because you have just exhausted the generator the first time around !

my_nums = (i*i for i in [1,2,3,4,5])
for k in my_nums:
    print(k)

my_nums = (i*i for i in [1,2,3,4,5])
print(list(my_nums))

Answer 3

When you are iterating over the generator, you empty it at the same time. So, just remove for k in (my_nums): print(k) and you will have the expected result.

Same thing here:

my_nums =(i*i for i in [1,2,3,4,5])
a = list(my_nums)
b = list(my_nums)
print(a)
// [1,4,9,16,25]
print(b)
// []

Answer 4

That's because you have already exhausted all the generated values when you did

for k in (my_nums): # you already made my_nums generate till the end here
    print(k)

Think of the generator as reading from a stream. You have already read the stream and so you get an empty list when you try to read more from an already depleted generator

Answer 5

A generator expression like this:

my_nums = (item for item in [1, 2, 3, 4, 5, 6, 7])

is syntax sugar for something like this:

def generator(iterable):
    for item in iterable:
        yield item

my_nums = generator([1, 2, 3, 4, 5, 6, 7])

The forloop itself is equivalent to something like this:

while True:
    try:
        item = next(my_nums)
    except StopIteration:
        break
    else:
        print(item)

Since gg = list(iterable) is something like:

gg = []
for item in my_nums:
    gg.append(item)

The problem is that the first time you loop over the generator expression, it is exhausted. If you try to loop over it again, it will just raise StopIteration and break the loop.

For example:

>>> my_nums = (i for i in [1, 2])
>>> list(my_nums)
[1, 2]
>>> list(my_nums)
[]
>>> next(my_nums)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>>

See how the second time you call list(my_nums) it returns an empty list? This is because the generator is exhausted. You can see how it raises StopIteration if you try to fetch the next item from it.

Answer 6

Generators are iterators, but you can only iterate over them once. It’s because they do not store all the values in memory, they generate the values on the fly.

my_nums =(i*i for i in [1,2,3,4,5])
for k in (my_nums):
    print(k)

It prints:

1
4
9
16
25

It is just the same except you used () instead of []. BUT, you can not perform for k in my_nums a second time since generators can only be used once: they calculate 1, then forget about it and calculate 4, and end calculating 25, one by one.

If you try to perform the iteration again, you will get StopIteration traceback.

This has been discussed in detail on this thread.