3

I would like to know how much memory is being occupied by a variable.

Let us assume this example:

import numpy as np

t = [np.random.rand(10000) for _ in range(1000)]

How much memory in megabytes is t occupying?

I ran:

sys.getsizeof(array_list)

and I am getting 8856 which seems low for the amount of information contained in t

Best Regards.

Talha Tayyab
  • 8,111
  • 25
  • 27
  • 44
nop nop
  • 83
  • 6
  • This is a lot harder to pin down in languages like Python than, say, C. Are you counting the overhead of the array? – tadman May 17 '23 at 15:03
  • @tadman should be the size of everything. I am using `sys.getsizeof(t)` but it is giving me 8856 bytes... which seems too low – nop nop May 17 '23 at 15:06
  • That could be the size of the array, excluding the size of the values. In other words, also count the `getsizeof(e)` for each element `e`. – tadman May 17 '23 at 15:07
  • Yeah, getsizeof is telling you the size of the list, not its contained objects. – tdelaney May 17 '23 at 15:07
  • I would like the size of everything. How much memory am I alocating to store that array – nop nop May 17 '23 at 15:07
  • In this simple case, you could sum the getsizeof of the contents of the list (plus the list itself). `sum(sys.getsizeof(v) for v in t) + getsizeof(t)`. But the linked question above has more options. – tdelaney May 17 '23 at 15:08
  • you made a `list`, not an array. Lists contain pointers/references, so `getsizeof` is a poor measure. For numeric dtype arrays, `nbytes` and `getsizeof` are in good agreement. – hpaulj May 17 '23 at 16:05
  • For lists and dict total memory use has to take into account space used by the referenced elements, which can get very complicated. A list may have repeated references to the same object, ints of various sizes, floats, strings, and who knows what else. In comparison memory use of numeric dtype arrays is simple - if you take time to learn numpy basics. – hpaulj May 17 '23 at 16:23
  • A list of 1000 arrays, each with `10000` element floats - nbytes `of 10000*8, about 80000. So roughly 1000*8 for list references plus 1000*80000. for content. Here contents dominates to total memory use. 80Mb – hpaulj May 17 '23 at 16:31

3 Answers3

3
from pympler import asizeof
asizeof.asizeof([np.random.rand(10000) for _ in range(1000)])

#output
80120872

Link : https://pypi.org/project/Pympler/

pip install Pympler
Talha Tayyab
  • 8,111
  • 25
  • 27
  • 44
  • I have tried this one. It works the best for list of multidimensional arrays. [Although it is an external library] – nop nop May 17 '23 at 15:38
1

I guess this (without resorting to external libraries) does the trick:

def deepgso(ob):
    size = sys.getsizeof(ob)
    if isinstance(ob, (list,tuple,set)):
        for element in ob:
            size+=deepgso(element)
    if isinstance(ob, dict):
        for k,v in ob.items():
            size+=deepgso(k)
            size+=deepgso(v)
    return size
nop nop
  • 83
  • 6
-1

Short answer: sys.getsizeof(variable)

Example:

import sys
integer = 10
print("The size of the integer variable is:",sys.getsizeof(integer), "bytes.")

Full answer

Konstantin Z
  • 146
  • 7