pass different "C" functions with pointer arrays as the function argument to a class

Question

I am trying to pass different functions which have pointers as arguments to a python function. One example of the input function as input parameter is the given normal function:

Sample.pyx

from cpython cimport array
import cython
import ctypes
cimport numpy as np
cpdef void normal(np.ndarray[ndim=1, dtype=np.float64_t] u, 
                  np.ndarray[ndim=1, dtype=np.float64_t] yu, 
                  np.ndarray[ndim=1, dtype=np.float64_t] ypu):

      cdef int i
      cdef int n=len(u)
      for i in prange(n, nogil=True):
          yu[i]=-u[i]*u[i]*0.5                                                               
          ypu[i]=-u[i]                                                                    
      return                                                     
cdef class _SampleFunc:
     cdef void (*func)(double *, double *, double *)

cdef void sample(int* x, double* hx, double* hxx, void(*func)(double*, double*, double*), int n):
      def int i
      for i from 0 <= i < n:
          func[0](&x[i], &hx[i], &hxx[i])
      return 
cdef class myClass:
     sample_wrapper = _SampleFunc() 
     sample_wrapper.func = Null
     def foo(np.ndarray[ndim=1, dtype=np.float64_t] x,
             np.ndarray[ndim=1, dtype=np.float64_t] hx,
             np.ndarray[ndim=1, dtype=np.float64_t] hxx,
             _SampleFunc sample_func, 
             int k):
         cdef np.ndarray[ndim=1, dtype=np.float64_t] sp
         cdef int num=len(x)
         func = sample_func.func
         assert func is not NULL, "function value is NULL"
         cdef int j
         for j from 0 <= j <k:
             sample(&x[0],&hx[0], &hxx[0], func, num)
             sp[j]=hx[0]
         return sp

test.py

import numpy as np
from sample import *
x = np.zeros(10, float)
hx = np.zeros(10, float)
hpx = np.zeros(10, float)

x[0] = 0
x[1] = 1.0
x[2] = -1.0
def pynormal(x):
    return -x*x*0.5,-x

hx[0], hpx[0] = pynormal(x[0])
hx[1], hpx[1] = pynormal(x[1])
hx[2], hpx[2] = pynormal(x[2])
num=20
ars=myClass()
s=ars.foo( x, hx, hpx, normal, num)

Running the test.py code I am getting this error:

'ars._SampleFunc' object has no attribute 'func'

I am trying to write a wrapper for different C functions which have three pointer arrays as their argument. My conclusion so far was that it can be done with a class, since the class can be accessible in python. I am wondering how I can pass the C functions with pointer arrays as argument to myClass class?

Update: Normal function

cdef void normal(
                 int n,
                 double* u, 
                 double* yu, 
                 double* ypu
                ):          
      cdef int i          
      for i in prange(n, nogil=True):
          yu[i]=-u[i]*u[i]*0.5                                                               
          ypu[i]=-u[i]                                                                    
      return 

Answer 1

The first thing to deal with is that a function of signature cdef void (*func)(double *, double *, double *) does not pass the array length. You can't know how long these arrays are, and thus you can't safely access their elements. The sensible thing is to change the function signature to pass a length too:

cdef void (*func)(double *, double *, double *, int)

What is extra confusing is that you seem to be iterating over the same axis of a 1D array in both normal and sample. I suspect that isn't what you want to do, but I'm not going attempt to fix that.

---

Essentially your problem is that you want to pass an arbitrary Python callable as a C function pointer. The bad news is that Cython can't do it - a Python callable has a significant amount of information associated with it, while a C function pointer is simply the address of some executable memory. Therefore a C function pointer does not have the space available to hold the information in a Python callable. In order to make this work you need to generate code at runtime, which Python can't do.

I've recommended the ctypes standard library module as a solution to similar problems previously, since it can create a function pointer from a Python callable. There is a simpler but more limited solution if you only want to call cdef Cython functions.

ctypes

Here's a minimal example which demonstrates how to implement the idea:

import numpy as np
import ctypes

ctypedef void (*func_t)(int, double *)

cdef void sample(int n, double* x, func_t f):
    f(n,x)

def call_sample(double[::1] x,
                f):

    def func_wrapper(n, arg1):
        # x is a slightly opaque ctypes type
        # first cast it to a ctypes array of known size
        # and then create a numpy array from that
        arg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))
        return f(np.asarray(arg1_as_ctypes_array))


    FTYPE = ctypes.CFUNCTYPE(None, # return type
                             ctypes.c_int, # arguments
                             ctypes.POINTER(ctypes.c_double))
    f_ctypes = FTYPE(func_wrapper) # convert Python callable to ctypes function pointer

    # a rather nasty line to convert to a C function pointer
    cdef func_t f_ptr = (<func_t*><size_t>ctypes.addressof(f_ctypes))[0]

    sample(x.shape[0], &x[0], f_ptr)


def example_function(x):
    # expects a numpy array like object
    print(x)

def test():
    a = np.random.rand(20)
    print(a)
    call_sample(a,example_function)

I realise that there's some slightly messy conversion between ctypes and Cython - this is unavoidable.

A bit of explanation: I'm assuming you want to keep the Python interface simple, hence example_function just takes a numpy array-like object. The function passed by ctypes needs to accept a number of elements and a pointer to match your C interface.

The ctypes pointer type (LP_c_double) can do do indexing (i.e. arg1[5]) so it works fine for simple uses, but it doesn't store its length internally. It's helpful (but not essential) to change it to a numpy array so you can use it more generally and thus we create a wrapper function to do this. We do:

arg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))

to convert it to a known length ctypes array and then

np.asarray(arg1_as_ctypes_array)

to convert it to a numpy array. This shares the data rather than makes a copy, so if you change it then your original data will be changed. Because the conversion to a numpy array follows a standard pattern it's easy to generate a wrapper function in call_sample.

(In the comments you ask how to do the conversion if you're just passing a double, not a double*. In this case you don't have to do anything since a ctypes double behaves exactly like a Python type)

Only cdef functions

If you're certain the functions you want to pass will always be cdef functions then you can avoid ctypes and come up with something a bit simpler. You first need to make the function signature match the pointer exactly:

cdef void normal(int N, double *x): # other parameters as necessary
    cdef double[::1] x_as_mview = <double[:N:1]>x # cast to a memoryview
    # ... etc

You should then be able to use your definition of SampleFunc almost as is to create module level objects:

# in Cython
normal_samplefunc = SampleFunc()
normal_samplefunc.func = &normal

# in Python
s=ars.foo( x, hx, hpx, normal_samplefunc, num)

ars.foo is the way you wrote it (no ctypes code):

func = sample_func.func
# ...
sample(..., func,...)

This code will run quicker, but you want be able to call normal from Python.

---

Python interface

You mention in the comments that you'd also like the be able to access normal from Python. You're likely to need a different interface for the Python function and the one you pass to C, so I'd define a separate function for both uses, but share the implementation:

def normal(double[::1] u, # ... other arguments
           ):
   # or cpdef, if you really want
   implementation goes here

# then, depending on if you're using ctypes or not:
def normal_ctypes(int n, u # other arguments ...
   ):
   u_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(x.contents))
   normal(u_as_ctypes_array, # other arguments
                )

# or
cdef void normal_c(int n, double* u # ...
              ):
   normal(<double[:N:1]>x # ...
          )