1

I'm trying to understand how much memory allocation behaves for dictionaries when deleting keys and adding new keys to them for the below example. (since this is a single long program I've grouped into sections- welcoming question edits if you felt non-readable)

Example:

d = {
    "one": 1,
    "two": 2,
    "three": 3
}

import sys
def get_size(obj):
    return sys.getsizeof(obj)

#section 1
print("d - without modification: ")
print(d)
print(get_size(d))

#section 2
print("d - after deletion of two: ")
del d["two"]
print(d)
print(get_size(d))

#section 3
print("d - after adding two_copy key: ")
d["two_copy"] = 2
print(d)
print(get_size(d))

#section 4
print("d - after adding few values modification: ")
d["three_copy"] = 3
d["four"] = 4
print(d)
print(get_size(d))

#section 5
import copy
d1 = copy.deepcopy(d)
print("d1 - after deepcopy of d: ")
print(d1)
print(get_size(d1))

Output:

d - without modification: 
{'two': 2, 'three': 3, 'one': 1}
288

d - after deletion of two: 
{'three': 3, 'one': 1}
288

d - after adding two_copy key: 
{'two_copy': 2, 'three': 3, 'one': 1}
288

d - after adding few values modification: 
{'two_copy': 2, 'three_copy': 3, 'four': 4, 'three': 3, 'one': 1}
288

d1 - after deepcopy of d: 
{'two_copy': 2, 'three': 3, 'one': 1, 'four': 4, 'three_copy': 3}
288

In the above example, section 1 is the normal one that we used to see. For section 2, why size here is same as before even removing a key from dictionary?. When we add new key to the section 3, it stays in the same size. But why adding two more new keys in the same dictionary not increasing its size? on section 4.

And when I add new key in section 4, the size of section 4 and section 5 returns different values like this,

print("d - after adding few values modification: ")
d["three_copy"] = 3
d["four"] = 4
d["five"] = 5    # this is the newly added key
print(d)
print(get_size(d))

Output for section 4 and section 5:

d - after adding few values modification: 
{'two_copy': 2, 'three_copy': 3, 'one': 1, 'four': 4, 'five': 5, 'three': 3}
480
d1 - after deepcopy of d: 
{'two_copy': 2, 'three_copy': 3, 'one': 1, 'five': 5, 'four': 4, 'three': 3}
480

Why size changes after adding one more new key on the same program?

I've marked all my questions on bold. Please anyone help me to understand what's happening under the hoods? Thanks.

You can run the same program here.

surya
  • 719
  • 5
  • 13
  • 3
    Container sizes (dict, list, set) do not change every time the amount of elements change. That would have a huge overhead and cause a performance hit (imagine if every time you added a key to a dict new memory was allocated and data had to be copied). Instead, Python uses a smart algorithm to determine when it is needed to change the memory allocated, and by how much. – DeepSpace Jun 09 '21 at 10:16
  • Nice, can I get some reference of what you're mentioning here - ```Python uses a smart algorithm to determine when it is needed to change the memory allocated, and by how much.``` – surya Jun 09 '21 at 10:19
  • Found well written answer here: https://stackoverflow.com/a/65512841/11094041 – surya Jun 09 '21 at 10:23
  • 1
    @say_my_name the C code has excellent documentation if you want to dive into that. https://github.com/python/cpython/blob/main/Objects/dictobject.c – Axe319 Jun 09 '21 at 10:53

0 Answers0