7

I am trying to run a Python (2.7) script with PyPy but I have encountered the following error:

TypeError: sys.getsizeof() is not implemented on PyPy.

A memory profiler using this function is most likely to give results
inconsistent with reality on PyPy.  It would be possible to have
sys.getsizeof() return a number (with enough work), but that may or
may not represent how much memory the object uses.  It doesn't even
make really sense to ask how much *one* object uses, in isolation
with the rest of the system.  For example, instances have maps,
which are often shared across many instances; in this case the maps
would probably be ignored by an implementation of sys.getsizeof(),
but their overhead is important in some cases if they are many
instances with unique maps.  Conversely, equal strings may share
their internal string data even if they are different objects---or
empty containers may share parts of their internals as long as they
are empty.  Even stranger, some lists create objects as you read
them; if you try to estimate the size in memory of range(10**6) as
the sum of all items' size, that operation will by itself create one
million integer objects that never existed in the first place.

Now, I really need to check the size of one nested dict during the execution of the program, is there any alternative to sys.getsizeof() I can use in PyPy? If not, how would I check for the size of a nested object in PyPy?

alec_djinn
  • 10,104
  • 8
  • 46
  • 71
  • 1
    "I really need to check the size of one nested dict during the execution of the program" - `sys.getsizeof` wouldn't have been the right tool for that even under CPython. It doesn't consider the sizes of other objects an object holds references to, such as dict keys and values. – user2357112 Mar 02 '17 at 19:19
  • It's simple, you can implement a function that iterates over the nested dict and compute the cumulative size of the whole object. http://stackoverflow.com/questions/449560/how-do-i-determine-the-size-of-an-object-in-python – alec_djinn Mar 02 '17 at 19:22
  • 4
    I suppose people will still ask for ``sys.getsizeof()`` even if the wall of text explains in even more details why it doesn't make sense to, complete with examples. – Armin Rigo Mar 03 '17 at 22:25
  • @ArminRigo It is very clear, but still, there must be a 'good enough' way of checking the size of an object. – alec_djinn Mar 12 '17 at 08:59
  • It seems just absurd to me that there is no way of getting the size of an object in PyPy. – alec_djinn Mar 21 '17 at 16:12

1 Answers1

3

Alternatively you can gauge the memory usage of your process using 

import resource
resource.getrusage(resource.RUSAGE_SELF).ru_maxrss

As your program is executing, getrusage will give total memory consumption of the process in number of bytes or kilobytes. Using this information you can estimate the size of your data structures, and if you begin to use say 50% of your machine's total memory, then you can do something to handle it.

goCards
  • 1,388
  • 9
  • 10
  • This is a way to manage memory consumption of the whole process, but it does not give me information on the memory allocated for a specific object. It can work for my specific case. Thanks. – alec_djinn Mar 14 '17 at 09:03