3

I recall reading that it is hard to pin down the exact memory usage of objects in Python. However, that thread is from 2009, and since then I have read about various memory profilers in Python (see the examples in this thread). Also, IPython has matured substantially in recent months (version 1.0 was released a few days ago)

IPython already has a magic called whos, that prints the variable names, their types and some basic Data/Info.

In a similar fashion, is there any way to get the size in memory of each of the objects returned by who ? Any utilities available for this purpose already in IPython?

Using Guppy

Guppy (suggested in this thread) has a command that allows one to get the cummulative memory usage per object type, but unfortunately:

  1. It does not show memory usage per object
  2. It prints the sizes in bytes (not in human readable format)

For the second one, it may be possible to apply bytes2human from this answer, but I would need to first collect the output of h.heap() in a format that I can parse.

But for the first one (the most important one), is there any way to have Guppy show memory usage per object?

In [6]: import guppy

In [7]: h = guppy.hpy()                                                                                                                                     

In [8]: h.heap()
Out[8]: 
Partition of a set of 2871824 objects. Total size = 359064216 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0 522453  18 151469304  42 151469304  42 dict (no owner)
     1 451503  16 36120240  10 187589544  52 numpy.ndarray
     2 425700  15 34056000   9 221645544  62 sklearn.grid_search._CVScoreTuple
     3 193439   7 26904688   7 248550232  69 unicode
     4 191061   7 22696072   6 271246304  76 str
     5 751128  26 18027072   5 289273376  81 numpy.float64
     6  31160   1 12235584   3 301508960  84 list
     7 106035   4  9441640   3 310950600  87 tuple
     8   3300   0  7260000   2 318210600  89 dict of 0xb8670d0
     9   1255   0  3788968   1 321999568  90 dict of module
<1716 more rows. Type e.g. '_.more' to view.>
Community
  • 1
  • 1
Amelio Vazquez-Reina
  • 91,494
  • 132
  • 359
  • 564
  • ... wait: you want to know the size of each one of the `522453` `dict` instances, `451503` `numpy.ndarray` instances etc? AFAIK there isn't any library/profiler doing this. I'd also point out that, for example, information on `numpy.ndarray` will be wrong since `numpy` does not use `PyMem_MALLOC`, but uses `malloc` directly(hence every non-numpy aware profile wont be able to correctly inspect the size of the arrays). – Bakuriu Aug 20 '13 at 17:16
  • @Bakuriu, not quite, I would like to know the size of each of the objects in the current frame without recursion (i.e. those listed by `who` in IPython). Just updated the OP to clarify this. – Amelio Vazquez-Reina Aug 20 '13 at 17:30

1 Answers1

3

Why not use something like:

h.heap().byid

But this will only show you immediate sizes (i.e. not the total size of a list including the other lists it might refer to).

If you have a particular object you wish to get the size of you can use:

h.iso(object).domisize

To find the approximate amount of memory that would freed if it were deleted.

Anon
  • 6,306
  • 2
  • 38
  • 56