-2

I recently came across this great stackoverflow post explaining the concept of yield What does the "yield" keyword do in Python?.

As such I did some exploring myself and found that the size of a generator, although substantially smaller than that of an iterable (list in this case) that the generator still holds a very small chunk of memory.

Whats more interesting to me is if i increase the range(100) -> range(10000), and add in intermediate variables e.g. test = i * 100 + 2 ... yield test, that the memory usage of b remains a constant 112 bytes, while that of a increases as I would expect. Is this memory independent of the size of the function referenced for delayed execution in the generator?

from sys import getsizeof

a = [i for i in range(100)]

def b(): 
    for i in range(100):
        yield i
        
b = b()

getsizeof(a) # 920
getsizeof(b) # 112

So the question is - what does b's memory actually hold?

ZooPanda
  • 331
  • 3
  • 11
  • 1
    `b` is just an interrupted function. When you do the `yield`, the "stack frame" for the function is essentially frozen, and the function returns. When the caller does something that triggers a `next`, the function is unfrozen and continues to execute as if it has never stopped. There's no extra memory. Your variable `b` does not "contain" the list. It contains a reference to the interrupted function. – Tim Roberts Nov 12 '22 at 01:10
  • 2
    That's not a good test. `range` objects don't hold anything except like 3 numbers, so increasing the "size" of the range doesn't affect the size of the object. `a` grows because putting the range in a list forces each number to be realized because lists actually hold elements. – Carcigenicate Nov 12 '22 at 01:11
  • 2
    `sys.getsizeof` is simply not that useful of a function, since it only tells you the size of the object itself - not any other objects that are referred to by it. So the size of `a` depends solely on the number of elements it contains - and not the complexity of those elements; the size of `b` is probably constant for all defined functions, and tells you nothing about the complexity of the function (that's all contained in sub-objects). It *certainly* does not tell you anything about the runtime memory use of the function. – jasonharper Nov 12 '22 at 01:14

1 Answers1

0

a with a list comprehension is building and holding the entire object in memory.

b is a generator and it doesn't build nor hold the entire object in memory. Instead, it keeps state and runs up to yield every time you call next().

LorenzoPeve
  • 23
  • 4