1

I want to access some python functions that I wrote from my C code. One of my python functions receives a numpy array as an input. In my C code, this array is allocated in the dynamic memory. Whenever my python function is called I receive a segmentation fault, and I am not sure what am I doing wrong. I tried to pass N, O, O& to the Py_BuildValue function, but I keep on getting the same result.

Here is the python simplified code.

#Python simple function that receives pointer to array
#!/usr/bin/python3

import numpy as np
def getNumpyArrayPointer(pointer):
    data = np.ctypeslib.as_array(pointer, shape=(1000,1)).astype(np.int32) #Doesn't matter if I comment out this line. Fault continues
    print('Python function getNumpyArrayPointer() called')
    return data

Here is the C simplified code

#include <iostream>
#include <Python.h>
#include "pyhelper.h"


int capitest()
{
    Py_Initialize();
    PyObject* sysPath = PySys_GetObject((char*)"path");
    PyObject* programName = PyUnicode_FromString("path-to-python-script");
    PyList_Append(sysPath, programName);
    Py_DECREF(programName);
    PyObject* pName = PyUnicode_FromString("python-script-name");
    PyObject* pModule = PyImport_Import(pName);
    PyObject* pValue;
    CPyObject pFunc;
    PyObject* args;

    if(pModule)
    {
        pFunc = PyObject_GetAttrString(pModule, "getNumpyArrayPointer");
        uint32_t *array = (uint32_t *)malloc (1000);
        args = Py_BuildValue("(N)", array );
        
        if(pFunc && PyCallable_Check(pFunc))
        {
            printf("Calling getNumpyArrayPointer\n");
            pValue = PyObject_CallObject(pFunc, args);
            Py_DECREF(args);
            printf("Called getNumpyArrayPointer\n");
            if (PyErr_Occurred()) 
            { 
                PyErr_Print(); 
                return 0; 
            }   
        }
        else
        {
            printf("ERROR: function getNumpyArrayPointer()\n");
        }
    }
    else
    {
        printf("ERROR: Module not imported\n");
    }
    Py_Finalize();
    return 0;
}

int main()
{
    capitest();
}

Output:

Calling getNumpyArrayPointer
Segmentation fault (core dumped)
Dawood
  • 11
  • 3

1 Answers1

-2
 uint32_t *array = (uint32_t *)malloc (1000);
 args = Py_BuildValue("(N)", array );

That's nonsense. You're telling it that array is a pointer to a Python object so it'll try to manage the "reference count" of the array.

I'd suggest something along the lines:

PyObject* pointer_as_int = PyLong_FromVoidPtr(array);

You then want to pass this to Python object to ctypes.cast(pointer_as_int, ctypes.POINTER(ctypes.c_uint32)) to get your ctypes pointer. See for example ctypes: construct pointer from arbitrary integer

Alternatively you could try passing the C pointer directly the Numpy C API function PyArray_SimpleNewFromData - you'll find plenty of examples on Stack Overflow and elsewhere.

Note that your C code is responsible for freeing the array once you're done with it.

DavidW
  • 29,336
  • 6
  • 55
  • 86
  • Thanks! On the C side I added the lines `PyObject* number = PyLong_FromVoidPtr(array); args = Py_BuildValue("(O)", number );`. In the python script I changed it to `pyptr = ctypes.POINTER(ctypes.c_uint32).from_address(pointer) data = np.ctypeslib.as_array(pyptr, shape=(3072,3072))` Now I get a NULL pointer access Value error. Do you have any idea why? – Dawood Sep 05 '20 at 09:49
  • I think my conversion from int to a ctypes value was wrong. See the edit for something tested and working – DavidW Sep 05 '20 at 10:58
  • Thanks @DavidW. Just for references, the python code was the one crashing as I was converting the pointer to a np array without casting it. The way I am handling it right now is as follows `pyPointerObject = ctypes.cast(pointerFromC, ctypes.POINTER(ctypes.c_uint32)) npMatrix = np.ctypeslib.as_array(pyPointerObject, shape=(3072,3072))`. Thanks! – Dawood Sep 06 '20 at 10:00
  • I'm still fairly confident that crash was because you were misinterpreting a C array as a Python object (even if the crash only happened when you tried to use the object rather than at the moment you did the cast). Glad it's working now. – DavidW Sep 06 '20 at 10:08