0

I'm writing a Python C API and in my c-code I allocate a double array which I then wrap into a numpy array and send back to Python. The weird thing is that the memory is being freed in Python if I use PyMem_ functions to allocate the memory, but not if I use regular c-functions such as malloc/calloc, giving me segmentation faults or errors such as 'pointer being freed was not allocated'.

Example of c-function called from Python:

static PyObject* c_function_called_from_python(...){
    int nrElem;
    npy_intp dims[1];
    PyObject *array;
    double *data;

    dims[0] = nrElem;
    data = PyMem_Calloc(nrElem, sizeof(double)); // Does not work
    // data = calloc(nrElem, sizeof(double)); // Do work
    array = PyArray_SimpleNewFromData(1, dims, NPY_DOUBLE, (void *) data);
    PyArray_ENABLEFLAGS((PyArrayObject *) array, NPY_ARRAY_OWNDATA);
    
    return array;
}

Is there something special about PyMem_ functions that cause the memory to be freed?

Gustav Magnusson
  • 1
  • 2
  • 1
    This answer is for cython: https://stackoverflow.com/a/55959886/5769463 but it explains what is going on and shows how to do it correctly (which should be easily translated from cython) – ead Dec 07 '20 at 16:38
  • Since you're building a Numpy array, you could consider using the [C API](https://numpy.org/devdocs/reference/c-api/array.html#c.PyArray_Zeros) directly to allocate your array. – bnaecker Dec 07 '20 at 16:57
  • As I understand the solution involves creating a wrapper object that owns and frees the data. But what I don't understand is how PyMem_Malloc/Calloc knows that it is being owned by a object? The data comes from a c function which do not beforehand know the size of the array, so using the numpy C api is not optimal. – Gustav Magnusson Dec 08 '20 at 08:20
  • The point is that if the memory was allocated with `malloc` it _must_ be freed with `free` and not `PyMem_Free` (because they do somewhat different things). Your problem is that the Numpy array uses `PyMem_Free` (and thus it fails) – DavidW Dec 13 '20 at 21:25
  • @DavidW Yes I realized this, although I think it is the other way around, since my program crashed when I allocated memory using the PyMem_Calloc-function but not when using the regular calloc-function. Hence, I believe Numpy uses the regular free-function and not the PyMem_Free-function. – Gustav Magnusson Jan 19 '21 at 15:09

0 Answers0