size of nested dictionary and list

Question

I have dictionary with following format:

Demo code:

>>> import pprint
>>> pprint.pprint(data)
{'lookup': {'F01': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F02': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F03': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F04': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F05': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F06': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F07': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F08': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F09': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F10': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F11': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }',
            'F12': '\n.custom1 {\n    background-color: #f5e9dc;\n    padding: 10px;\n    border-radius: 10px;\n    font-family: sans-serif;\n    font-size: 0.9em;\n    margin-top: 1em;\n    }\n.custom2 .style8-rw {\n    font-family: sans-serif;\n    font-weight: bold;\n    color: #F57215;\n    }'},
 'sequence': ['F01',
              'F02',
              'F03',
              'F04',
              'F05',
              'F06',
              'F07',
              'F08',
              'F09',
              'F10',
              'F11',
              'F12']}
>>> import sys
>>> sys.getsizeof(data)
136
>>> sys.getsizeof(data["sequence"])
80
>>> sys.getsizeof(data["lookup"])
520
>>>

I can not get how nested dictionary is store in memory because size if data is 136 bytes and size of data["sequence"] is 80 bytes and also size of data["lookup"] is 520 bytes.

Also when I do type casting of variable data from dictionary to string then size of string variable is 3587 bytes.

Demo Code:

>>> data_str = str(data)
>>> sys.getsizeof(data_str)
3587

Can any on explain me why?

related: https://stackoverflow.com/q/449560 – djvg Jun 05 '23 at 09:30 — djvg, Jun 05 '23 at 09:30

score 12 · Accepted Answer · edited May 23 '17 at 12:32

Dictionaries and lists store references (like every other standard container in Python). sys.getsizeof() doesn't follow references, it gives you the memory footprint of the C structure only. The references are C pointers; their size depends on your specific platform.

Converting a dictionary to a string recursively converts the contents to (repr()) strings too, so all those references are then dereferenced and included in the output. Note that this is not an accurate reflection of the memory size for the original object; strings contain characters, it depends on your exact Python version, OS and range of Unicode codepoints used how much memory each character takes, and the character count has a non-linear relationship to the actual object being reflected.

If you want to know the memory footprint of a dictionary with the contents you need to do so recursively. Take into account that a dictionary can contain references to itself (directly or indirectly), or that any object can have multiple references to it and should only be counted once. I'd use the id() function to track what objects have already been handled.

There are already several posts on Stack Overflow that discuss calculating memory size for containers using recursion or other tools, see Deep version of sys.getsizeof, Python deep getsizeof list with contents?, and Memory-usage of dictionary in Python? for some examples.

1. yes, Dictionary store references. how to get total size of nested dictionary? or we can get size by type casting to string? — Vivek Sable, Feb 11 '16 at 12:05
@VivekSable: I already answer that. Recurse, but take into account that you can have circular references and objects stored multiple times. For example, a Python process only ever has **one** `None` object, so don't count it multiple times when you have `{'foo': None, 'bar': None, 'baz': None}`. — Martijn Pieters, Feb 11 '16 at 12:06
@VivekSable: and using `str()` is very far from an accurate reflection of the memory use. — Martijn Pieters, Feb 11 '16 at 12:08
I not refresh answer. Yes, `str()` is not best way to findout size of nested dictionary. — Vivek Sable, Feb 11 '16 at 12:17

size of nested dictionary and list

1 Answers1

Linked

Related