4

I have a long running process that allocates and releases objects constantly. Although objects are being freed, the RSS mem usage goes up over time.

How can I calculate how much fragmentation is happening? One possibility is to calculate RSS / sum_of_allocations and take that as an indicator. even then, how to do I calculate the denominator (sum_of_allocations).

Ali
  • 1,503
  • 2
  • 15
  • 20
  • 1
    why do you think it is a memory fragmentation issue? Try [Python memory profiler](http://stackoverflow.com/questions/110259/python-memory-profiler) – jfs Dec 09 '12 at 03:54
  • @J.F.Sebastian I was going to do that next. But Python is known to fragment memory for long lived processes that allocate/release small chunks of memory. – Ali Dec 10 '12 at 00:03
  • @J.F.Sebastian So using the profiler, how do you figure if fragmentation is happening or not ? Do you just sum the memory allocations and compare that with RSS ? – Ali Dec 10 '12 at 00:05
  • Memory profiler allows you to find a memory leak (an alternative explanation to "RSS mem usage goes up"). – jfs Dec 10 '12 at 07:47
  • 1
    Did you manage to solve this in the end? I'm in a similar boat. – CadentOrange Oct 18 '13 at 13:08

1 Answers1

1

Check out the Garbage Collector interface, gc.

http://docs.python.org/2/library/gc.html

You can inspect the objects are being tracked with gc.get_objects()

"As a general rule, instances of atomic types aren’t tracked and instances of non-atomic types (containers, user-defined objects...) are."

There is also gc.garbage, which finds objects that can't be freed but are unreachable.

matt
  • 1,895
  • 5
  • 19
  • 26