2

I have the following C++ class :

.H

class ALabSet: public LabSet {
public:
    PyObject *m_obj;

    ALabSet(PyObject *obj);

    virtual ~ALabSet();

    PyObject *GetPyObj();

};

.CPP

ALabSet::ALabSet(PyObject *obj): LabSet() {

    this->m_obj = obj;
    // Provided by "cyelp_api.h"
    if (import_cyelp()) {
    } else {
        Py_XINCREF(this->m_obj);
    }

}


ALabSet::~ALabSet() {
    Py_XDECREF(this->m_obj);
}


PyObject *ALabSet::GetPyObj() {
    return this->m_obj;
}

I exposed it as follows with Cython :

cdef extern from "adapter/ALabSiteSetsManager.h" namespace "elps" :
    cdef cppclass ALabSet:
        ALabSet(PyObject *)

        PyObject *GetPyObj()



cdef class PyLabSet:
    cdef ALabSet *thisptr

    def __cinit__(self):
       self.thisptr = new ALabSet(<PyObject *>self)

    def __dealloc__(self):
       print "delete from PY !"
       if self.thisptr:
           del self.thisptr

My problem is that I can't figure out how to get the destructor called from Python. The following does exactly nothing :

a_set = PyLabSet()
del a_set

I can't find similar issues on the web. Does any of you has an idea of is appening to me here ?

I'm I missing something about reference counting management, or ...

Thanks a lot

Gauthier Boaglio
  • 10,054
  • 5
  • 48
  • 85
  • 3
    You are creating a reference cycle I think, in `self.thisptr = new ALabSet(self)` you are increasing the reference count of your class. – Wessie Apr 12 '13 at 15:04
  • Yep, magic you ! I removed the additional Py_XINCREF s, and it did the trick. Good thinking. Thanks. Now I do not remember well what I was doing in my constructor. Need to have a look at "import_cyelp()" which is apparently generated by Cython, but I don't know what it does anymore ? – Gauthier Boaglio Apr 12 '13 at 15:14
  • I encountered a similar issue, because I was creating cyclic references. Solved it by using [`weakref.proxy`](https://docs.python.org/3.7/library/weakref.html#weakref.proxy), which is nicely described [here](https://pymotw.com/2/weakref/). – 0 _ Jan 29 '17 at 13:54
  • Relevant: https://stackoverflow.com/q/9449489/1959808 – 0 _ Oct 08 '17 at 02:05

1 Answers1

5

del a_set removes a reference to the object (the local variable). There's still another reference, in the C++ object. This is known as a reference cycle. The cycle GC could collect this after a while. However, there is no guarantee when (or even if) this happens, so you should not rely on it1.

For example, reference cycles containing pure Python objects with a __del__ special method are documented to not be freed at all:

Changed in version 3.4: Following PEP 442, objects with a __del__() method don’t end up in gc.garbage anymore.

I don't know whether Cython's implementation of __dealloc__ triggers this behavior, but as outlined before, destruction isn't deterministic anyway. If you want to free some resource (e.g. a block of memory that isn't a Python object, a file, a connection, a lock, etc.) you should expose an explicit way of doing so manually (cf. the close methods of various objects). Context managers can simplify client code doing this.

Disclaimer: Almost all of this is CPython-specific.

1 Some people prefer thinking of GC as an abstraction that simulates availability of infinite memory, rather than something that destroys unreachable objects. With this approach, it becomes quite obvious that destruction is not deterministic and not even guaranteed.

0 _
  • 10,524
  • 11
  • 77
  • 109
  • Ok, thank for the useful information. I aware of the non-deterministic behaviour, but I had a huge memory leak because the I was creating twice the reference for each object as @Wessie suggested... – Gauthier Boaglio Apr 12 '13 at 15:23
  • 1
    @Golgauth Actually, in most cases being silent about a reference to "break" a cycle is wrong. It means the reference won't keep the object alive, so it may disappear under your feet, and unlike weak references (which are the proper solution if you care enough) you aren't told when it happens and have no way of checking validity of the reference. In your specific case it *might* be okay, as the lifetimes of the two objects seem to be completely identical. Still, be *very* careful with this. Keeping the reference cycle (or using a weakref) and offering an explicit disposal mechanism is better. –  Apr 12 '13 at 15:30
  • Yes your right. I can see that now. When I am using directly the returned object from python (without storing it in a variable - ex : Count(GetPyLabSet()), where GetPyLabSet() returns a PyLabSet object), it's been freed already. It gives a nice crash. It seems that my only alternative is to keep my previous Py_XINCREF and create an additional function to explicitely call Py_XDECREF... Unless one of you has a 3rd solution to my problem. Thanks again. – Gauthier Boaglio Apr 12 '13 at 16:12
  • I am using `PyWeakref_NewRef` when keeping the reference to the object, and finally the dealloc is called :) like this `this->m_obj = PyWeakref_NewRef(obj, NULL);` I still do the `Py_XINCREF`. – dashesy Sep 12 '15 at 23:12
  • just one more thing, for extension classes have to have `cdef object __weakref__` in the class so that we could create a weakref. – dashesy Sep 16 '15 at 16:34