PyObject_CallMethod leaking / seg fault

Question

I call a Python method from C and it leak / seg fault on PyObject_CallMethod after 10 hours.

Can you review my code? Where is my error? Missing somethings?

The C code:

PyObject *pFrameAsList = PyList_New(0);
...
for (int k=0; k<m_nbImages; ++k)
{
   PyObject *pFrameAsArray = toNDArray(frame);
   ...
   PyList_Append(pFrameAsList, pFrameAsArray);
   Py_DECREF(pFrameAsArray);
}
PyObject *pValue = PyObject_CallMethod(pInstance, "analyseFrame", "(O)", pFrameAsList);
...
Py_INCREF(pValue);
...
for (int k=0 ; k<max; k++ )
{
    PyObject *ptemp = PyList_GetItem(pValue,k);
}
Py_DECREF(pFrameAsList);
Py_DECREF(pValue);

And the Python code:

def analyseFrame(self, frame):
   results = self.model.analyse(np.asarray(frame))
   return list(results)

You're missing **error checks everywhere**! They're not "optional" with C-API... — Antti Haapala -- Слава Україні, Jun 15 '18 at 16:40
I.e. you **must** check the return value of `PyList_New`, `toNDArray`, `PyList_Append`, `PyObject_CallMethod` and those `max` times of `PyList_GetItem`; in that case it might have been that your code didn't crash but would nicely exit with a MemoryError. — Antti Haapala -- Слава Україні, Jun 15 '18 at 16:43
You are all right, check the return value/code est the best practice and it's done in our code but I remove the checks here to reduce the code. — oBe, Jun 18 '18 at 07:08
how can we verify that these error checks are correct when you're not showing them! — Antti Haapala -- Слава Україні, Jun 18 '18 at 17:21

score 1 · Answer 1 · edited Jun 18 '18 at 17:25

1

I believe that the problem is that you shouldn't have the following line:

Py_INCREF(pValue)

The reason is that PyObject_CallMethod has already incremented the reference count for you. If you do the extra Py_INCREF call the reference count for the given PyObject will never go to 0.

According to the documentation, PyObject_CallMethod returns a new reference.

To understand the meaning of "new reference", as opposed to "borrowed reference" look at the first paragraph of the Reference Count Details section of the documentation:

edited Jun 18 '18 at 17:25

Antti Haapala -- Слава Україні

129,958
22
279
321

answered Jun 15 '18 at 16:37

Tim Boddy

1,019
7
13

If I remove the _Py_INCREF(pValue)_, I seg fault directly. Thanks for the link, I will read it. – oBe Jun 18 '18 at 07:14
I did a review according new / borrowed references according your comment and link. I remove the Py_INCREF on all method returning borrowed references. I'am testing... – oBe Jun 18 '18 at 09:20
It doesn't seem correct to me, based on the documentation, that the caller would have to do a Py_INCREF on the return value from PyObject_CallMethod because that returned reference is considered "new", rather than "borrowed". Perhaps you are passing pValue as an argument to some function that is "stealing" the reference? The notion of "stealing" a reference is documented at that same reference-count-details link I posted in the answer. – Tim Boddy Jun 18 '18 at 15:47
As suggested in https://stackoverflow.com/questions/510406/is-there-a-way-to-get-the-current-ref-count-of-an-object-in-python/510411 you could also use sys.getrefcount(pValue) at various points in your code to see what the current reference count is at various places in your code. – Tim Boddy Jun 18 '18 at 15:48

PyObject_CallMethod leaking / seg fault

1 Answers1