Difference in memory consumption between list comprehension and generator expression

Question

I am trying to measure the memory difference between a list created using a list comprehension to that created using a generator expression.

from memory_profiler import profile


@profile
def squares_using_generators():
    for square in (x ** 2 for x in range(200000)):
        if (square % 100000000 == 0):
            print(square)


@profile
def squares_using_list_comprehensions():
    from sys import getsizeof
    l = [x ** 2 for x in range(200000)]
    print('list size {}'.format(getsizeof(l)))
    for square in l:
        if(square % 100000000 == 0):
            print(square)


if __name__=='__main__':
    squares_using_generators()
    squares_using_list_comprehensions()

Memory profile output of the above program is as follows

For squares_using_generators()

Line #    Mem usage    Increment   Line Contents
================================================
     4     37.9 MiB      0.0 MiB   @profile
     5                             def squares_using_generators():
     6     37.9 MiB      0.0 MiB       for square in (x ** 2 for x in range(200000)):
     7     37.9 MiB      0.0 MiB           if (square % 100000000 == 0):
     8     37.9 MiB      0.0 MiB               print(square)

For squares_using_list_comprehensions

Line #    Mem usage    Increment   Line Contents
================================================
    11     37.9 MiB      0.0 MiB   @profile
    12                             def squares_using_list_comprehensions():
    13     37.9 MiB      0.0 MiB       from sys import getsizeof
    14     45.1 MiB      7.2 MiB       l = [x ** 2 for x in range(200000)]
    15     45.1 MiB      0.0 MiB       print('list size {}'.format(getsizeof(l)))
    16     45.2 MiB      0.1 MiB       for square in l:
    17     45.2 MiB      0.0 MiB           if(square % 100000000 == 0):
    18     45.2 MiB      0.0 MiB               print(square)

My program also prints the list size in case of squares_using_list_comprehensions

list size 1671792

My question is difference in Memory in the two cases in 7.2MiB. Why is it not 1671792 Bytes

`getsizeof` only returns the size of the container itself, not including the objects in the container. — juanpa.arrivillaga, Mar 06 '17 at 08:31
@juanpa.arrivillaga - how do I get the size of the container + size of the objects — liv2hak, Mar 06 '17 at 08:32
Note: `28*len(l)+sys.getsizeof(l) == 7271792` gives you roughly the 7.2MiB. This is 28 (sizeof(int)) times the length of `l`. — juanpa.arrivillaga, Mar 06 '17 at 08:33
you need to do it recursively. This is addressed in the [docs](https://docs.python.org/3/library/sys.html#sys.getsizeof) — juanpa.arrivillaga, Mar 06 '17 at 08:34
because a python `int` is an object, so it takes more than 4 bytes. And actually, it can be more than 28, because python ints are arbitrarily sized. See [this](http://stackoverflow.com/a/10365639/5014455) answer. — juanpa.arrivillaga, Mar 06 '17 at 08:34
@juanpa.arrivillaga - you came up with number `28` by working backwards? — liv2hak, Mar 06 '17 at 08:36
no, try `sys.getsizeof(int())`, or `sys.getsizeof(42)` or whatever. — juanpa.arrivillaga, Mar 06 '17 at 08:36

Difference in memory consumption between list comprehension and generator expression

0 Answers0