2

I have written some C extension for my python programs, and I just noticed that there are some memory leakage problem, however, the C program itself won't leak memory, so I guess there is some problem in reference count. Currently when I use python console to run my program, after the computation is finished, the total memory of python3 is really big, indicating some objects are not released, is there anyway I can know what objects are there or when the objects are allocated?

The C extension is part of a big package, so it is impossible to paste the whole package here.

Ami Tavory
  • 74,578
  • 11
  • 141
  • 185
1a1a11a
  • 1,187
  • 2
  • 16
  • 25
  • 3
    `getrefcount()` may help if you know which objects are suspect of leaking https://docs.python.org/2/library/sys.html#sys.getrefcount – salezica Aug 27 '16 at 21:12
  • Thank you for your help! what ref count should I expect for a object at the end of computation? 1? (I am getting 2). Actually I have a class A, using C extension to do some computation, and store the results in it (for potential next computation), and I have some cleaning up method in __del__ method for cleaning in class A. It seems the destructor is not called until the end of script, why? Even if I `del object`, it is still not called. @slezica – 1a1a11a Aug 27 '16 at 22:39
  • 2 is the minimum. One reference you use to call `getrefcount`, the other reference is `getrefcount`'s own. In a python shell, `x = {}; sys.getrefcount(x)` gives `2`. This means the caller of `getrefcount` owns the only reference. If you lose that one, it should be gargabe collected. Are you loosing it? Are you sure C isn't leaking? I'd try `valgrind`. Also, `__del__` is not very reliable in `cpython`, though I don't remember the specifics – salezica Aug 27 '16 at 22:49
  • @slezica you are helping a lot, thank you! I didn't use valgrind over python, the suppress file seems not working well for python3. I did use valgrind on the C program itself (without passing reference to python3), there is no memory leakage problem. I just realized even if I call del object, gc.collect(), the __del__ still won't be called, it seems even if the object is not referenced any more, it sill won't be garbage collected. – 1a1a11a Aug 27 '16 at 23:21
  • Cyclic references may cause memory leaks, [old question](http://stackoverflow.com/questions/8025888/does-python-gc-deal-with-reference-cycles-like-this) [cpython doc](https://docs.python.org/3.5/c-api/gcsupport.html) – J.J. Hakala Aug 28 '16 at 11:04
  • thank you for pointing out, I have checked, I don't have cyclic reference @J.J.Hakala – 1a1a11a Aug 28 '16 at 17:05

0 Answers0