Python Tuple vs List vs Array memory consumption

Question

I've been reading Fluent code by Luciano Ramalho and in the chapter 'Overview of Built-in Sequences' when describing C struct behind float he states:

".. That's why an array of floats is much more compact than a tuple of floats: the array is a single object holding the raw values of a float, while the tuple consists of several objects - the tuple itself and each float object contained in it".

So I decided to confirm this and wrote a simple script.

import sys
import array

array_obj = array.array("d", [])
list_obj = []
tuple_obj = tuple()

arr_delta = []
tuple_delta = []
list_delta = []

for i in range(16):
    s1 = sys.getsizeof(array_obj)
    array_obj.append(float(i))
    s2 = sys.getsizeof(array_obj)
    arr_delta.append(s2-s1)
    
    s1 = sys.getsizeof(tuple_obj)
    tuple_obj = tuple([j for j in range(i+1)])
    s2 = sys.getsizeof(tuple_obj)
    tuple_delta.append(s2-s1)
    
    s1 = sys.getsizeof(list_obj)
    list_obj.append(i)
    s2 = sys.getsizeof(list_obj)
    list_delta.append(s2-s1)

print("Float size: ", sys.getsizeof(1.0))
print("Array size: ", sys.getsizeof(array_obj))
print("Tuple size: ", sys.getsizeof(tuple_obj))
print("List size: ", sys.getsizeof(list_obj))

print("Array allocations: ", arr_delta)
print("Tuple allocations: ", tuple_delta)
print("List allocations: ", list_delta)

Which produces the following output on my system (python 3.11.4, 64 bit):

Float size:  24
Array size:  208
Tuple size:  168
List size:  184
Array allocations:  [32, 0, 0, 0, 32, 0, 0, 0, 64, 0, 0, 0, 0, 0, 0, 0]
Tuple allocations:  [8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8]
List allocations:  [32, 0, 0, 0, 32, 0, 0, 0, 64, 0, 0, 0, 0, 0, 0, 0]

Allocations lists is how size of object was changing over time after appending new float (tuple recreates each time)

So, my questions are:
Aren't lists and tuples supposed to hold pointers to PyFloatObjects when arrays hold pointers to floats?
Or is it peculiarity of sys.getsizeof?
As can be seen in this SO question, C structs behind tuple and list contains arrays of pointers to PyFloatObject.

Thanks!

`array` objects contain the values directly. Each float value has the same size and meaning of its bytes, so the overhead of an object isn't needed. — Michael Butscher, Jul 21 '23 at 21:17
@MichaelButscher, yes I understand that and that is why I'm asking this question. According to sys.getsizeof, it seems that tuple and list also doesn't have this overhead. — Andrey, Jul 21 '23 at 21:20
@jonrsharpe Thanks! I missed this. I used [their recommended recipe](https://code.activestate.com/recipes/577504/) for getsizeof and now I can see that with each float tuple grow for 36 bytes. 24 bytes for PyFloatObject, 8 bytes for 64bit pointer, but what are the extra 4 bytes? — Andrey, Jul 21 '23 at 21:31
@juanpa.arrivillaga I replaced sys.getsizeof calls with [mentioned implementation](https://code.activestate.com/recipes/577504/) and according to it tuple grow in size with each float for 36 bytes. — Andrey, Jul 21 '23 at 21:44
@juanpa.arrivillaga My bad! I keep items as ints in tuple, so it really grow by 36 bytes with every int, but after casting them to floats, size growth came down to expected 32 bytes. — Andrey, Jul 21 '23 at 21:48

score 0 · Accepted Answer · answered Jul 22 '23 at 13:07

As @jonrsharpe pointed out in the comment section to my question, according to python docs:

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

So, for list sys.getsizeof returns size of list object itself plus 8 bytes (on 64-bit build) for every element, because it stores not elements but pointers to them.

The same python doc contains link to the recursive sizeof recipe. After replacing sys.getsizeof with suggested recursive sizeof I've got the following result

Array size:  208 208
Tuple size:  168 552
Array allocations:  [32, 0, 0, 0, 32, 0, 0, 0, 64, 0, 0, 0, 0, 0, 0, 0]
Tuple allocations:  [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32]

Array results stay the same, but tuple grow with each new element by 32 bytes = 24 bytes (PyFloatObject) + 8 bytes (64-bit pointer), which is expected.

Python Tuple vs List vs Array memory consumption

1 Answers1