3

The Python module tracemalloc offers a detailed insight into the allocated memory of the program. For example, one use case is to record the current and peak memory usage:

import tracemalloc

tracemalloc.start()

# ...code...

current, peak = tracemalloc.get_traced_memory()

If we now wanted the peak to be reset, the documentation suggests to use tracemalloc.reset_peak(). However, this function was only added in Python 3.9, and I was wondering whether I could achieve the same effect with tracemalloc.clear_traces()?

My use case is sth like this:

for i in range(10): 
     # do sth
     current, peak = tracemalloc.get_traced_memory()
     print('Current and peak memory usage: {} {}'.format(current, peak))
     # clear peak memory

So for every i in the for-loop, I do sth and want to measure only the memory of what I created. The peak should be only per index, not for the global run, that's why I want to clear it.

EDIT: To test the difference between reset_peak() and clear_traces(), I tested this program:

tracemalloc.start()
current_memories = []
peak_memories = []
for i in range(10): 
    a = list(range(100000))
    current, peak = tracemalloc.get_traced_memory()
    current_memories.append(current/(1024*1024))
    peak_memories.append(peak/(1024*1024))
    tracemalloc.reset_peak()
    # tracemalloc.clear_traces()
    del current, peak
print('Average current memory [MB]: {}, average peak memory [MB]: {} +/- {}'.format(
      round(np.mean(current_memories), 4), round(np.mean(peak_memories), 4), 
      round(np.std(peak_memories), 4))
)

When I test clear_traces(), the output is:

Average current memory [MB]: 3.4273, average peak memory [MB]: 3.4274 +/- 0.0019

When I instead use reset_peak(), I obtain:

Average current memory [MB]: 3.4313, average peak memory [MB]: 6.5156 +/- 1.0273

Why is there a difference between the two methods in the amount of memory they display?

Hermi
  • 350
  • 2
  • 5
  • 16
  • Why are you asking us? Have you tested it to see if it works yourself? – martineau Dec 29 '21 at 22:59
  • 1
    So, you just remember the last value you got, and subtract it from the new value. You don't need to clear anything. – Tim Roberts Dec 29 '21 at 23:05
  • @TimRoberts Not sure I can follow. After all, if I did two things and the memory consumption of the second operation would be much less than the first operation, then `peak` would still have the memory consumption of the first operation, right? – Hermi Dec 30 '21 at 09:15

1 Answers1

1

It seems impossible to mimic reset_peak before python3.9. reset_peak is a C function whose code is peak = current, and the memory counters are C variables, so it is not possible to modify them in python.

With clear_traces all previous allocations are forgotten:

>>> import tracemalloc
>>> tracemalloc.start()
>>> a = list(range(1000))
>>> tracemalloc.get_traced_memory()
(37619, 47929)
>>> tracemalloc.clear_traces()
>>> tracemalloc.get_traced_memory()
(8691, 19001)
Balaïtous
  • 826
  • 6
  • 9
  • I'd have a brief follow-up question: What exactly does it mean that "all previous allocations are forgotten"? Because judging from your example, it seems to me that we could use `tracemalloc.clear_traces()` instead of `tracemalloc.reset_peak()`, couldn't we? – Hermi Dec 30 '21 at 09:13
  • clear_traces set current to 0. After this only new allocation will be count. – Balaïtous Dec 30 '21 at 09:46
  • In the example that you provided, `current` is not `0`, though, is it? – Hermi Dec 30 '21 at 09:47
  • And what happens with `peak` when one calls `tracemalloc.clear_traces()`? – Hermi Dec 30 '21 at 09:48
  • Maybe some internal allocations for tracemalloc. – Balaïtous Dec 30 '21 at 09:51
  • clear_traces set peak to 0. – Balaïtous Dec 30 '21 at 09:54
  • Thanks! (-: Another thing that came up: When I call `tracemalloc.get_traced_memory()` and I get as output a tuple, in what units are the integers: Bits, bytes or something else? Unfortunately, the documentation does not specify this: https://docs.python.org/3/library/tracemalloc.html#tracemalloc.get_traced_memory. – Hermi Dec 30 '21 at 11:42
  • I also installed myself Python 3.9 in a separate environment to explicitly test `tracemalloc.reset_peak()`. I edited my question to show you the results. – Hermi Dec 30 '21 at 12:43