0

I have a problem with a project I am working on and am not sure about the best way to resolve it.

Basically I am pushing a slow python algorithm into a c++ shared library that I am using to do a lot of the numerically intense stuff. One of the c++ functions is of the form:

const int* some_function(inputs){

//does some stuff
int *return_array = new int[10];

// fills return array with a few values

return return_array;
}

I.e returns an array here. This array is interpreted within python using numpy ndpointer as per:

lib.some_function.restype = ndpointer(dtype=c_int, shape=(10,))

I have a couple of questions that I have been fretting over for a while:

1) I have dynamically allocated memory here. Given that I am calling this function through the shared library and into python, do I cause a memory leak? My program is long running and I will likely call this function millions of times, so this is important.

2) Is there a better data structure I can be using? If this was a pure c++ function I would return a vector, but from googling around, this seems to be a non- ideal solution in python with ctypes. I also have other functions in the c++ library that call this function. Given that I have just written the function and am about to write the others, I know to delete[] the returned pointer after use in these functions. However, I am unsatisfied with the current situation, as if someone other than myself (or indeed myself in a few months) uses this function, there is a relatively high chance of future memory leaks.

Thanks!

user3684792
  • 2,542
  • 2
  • 18
  • 23

2 Answers2

1

Yes, you are leaking memory. It is not possible for the Python code to automatically free the pointed-to memory (since it has no idea how it was allocated). You need to provide a corresponding de-allocation function (to call delete[]) and tell Python how to call it (possibly using a wrapper framework as recommended by @RichardHidges).

nobody
  • 19,814
  • 17
  • 56
  • 77
  • thanks, do you have an example of this behaviour? Is it only dynamically allocated objects that need to be python deallocated? – user3684792 Dec 10 '14 at 20:44
  • Why not simply pass an array as a parameter? That way the library doesn't have to worry about allocating and freeing memory. – Eryk Sun Dec 10 '14 at 22:21
  • @eryksun That could work, if the size is known to the caller ahead of time. – nobody Dec 10 '14 at 23:06
  • It's simpler with a fixed-size array. If the size is variable, there can be a 2nd parameter for the array size, measured in bytes or elements. In a two-pass call, the caller passes 0 for the size in order to query the required size, which is typically the function return value. – Eryk Sun Dec 10 '14 at 23:13
0

You probably want to consider using either SWIG or boost::python

There's an example of converting a std::vector to a python list using boost::python here: std::vector to boost::python::list

here is the link for swig: http://www.swig.org

Community
  • 1
  • 1
Richard Hodges
  • 68,278
  • 7
  • 90
  • 142