6

I have a long-running CPython 3.8 process. After a while, it's using a huge amount of RAM. I have tried

  1. running gc.collect()
  2. using pympler to discover all known Python objects
import gc
import psutil
from pympler import muppy, summarize

gc.collect()
total_ram = psutil.Process().memory_info().rss
all_objects = muppy.get_objects(include_frames=True)
s = summary.summarize(all_objects)
python_objects_size = sum(row[2] for row in s)

Output: 102 MiB Python objects, 824 MiB RSS memory!

[EDIT] 3. using tracemalloc; which also returns ~100MiB worth of python objects

[EDIT 2] export PYTHONMALLOC=malloc does not solve the problem.

Is there a way to query the CPython memory manager to figure out

  • How much RAM it is holding, so that I can subtract it from the RSS and figure out if there is any C library that is not using PyMem_Malloc and is leaking
  • Why it is holding the memory (e.g. find out that it's holding a 64kb page because of a single 20 bytes PyObject that's still being referenced)
  • Identify C modules that invoked PyMem_Malloc and never released the memory afterwards
  • Track the OS-level malloc() and free() and cross-compare them with the ones performed by pymalloc, to figure out if there's a C library that's allocating memory not with PyMem_Malloc

Related

crusaderky
  • 2,552
  • 3
  • 20
  • 28
  • 1
    https://stackoverflow.com/questions/552744/how-do-i-profile-memory-usage-in-python – RafalS May 06 '20 at 15:15
  • tracemalloc emits information totalling about 100MB - roughly the same reported by pympler. memory_profiler is giving me vastly underestimated results; e.g. if I allocate a list of 100k random floats, memory_profiles says that the line consumed 0.3 MiB while at the same time it reports that the RSS jumped up 3.8 MiB. – crusaderky May 20 '20 at 09:18
  • 1
    Not really an answer ... we had a similar issue at my work a couple of months ago. After spending some time with garbage collection analyzers, we eventually solved the issue using the good-old divide and conquer method: deleted some code, see if the issue persists, than delete some more, etc. etc. etc. (and a couple of more etc. after that - but it allowed us to resolve the issue eventually). – Roy2012 May 23 '20 at 19:11

1 Answers1

0

Maybe try a different memory debugger. It is possible to use valgrind with python.

Charles
  • 97
  • 5