2

I'm using ctypesgen and ctypesgen_to_pxd to generate both a ctypes wrapper for a given native library as well as a pxd containing those types.

I'm able to use the generated ctypes wrapper to call into native code from Python very easily. What I can't figure out how to do is to return a function pointer of a Python callback back via a hand-written pyx (see below) to my native library, which can then call that function pointer to execute some Python code.

Here is a complete and runnable sample on Github that shows my problem. You can build it using the provided build-and-run.sh or use VS Code remote containers to open the workspace. This latter method also allows you to debug and recompile as opposed to using the shell script = docker build + run.

The contents of cythoncallbacks.pyx.in is below for quick reference

# cython: language_level=3

import sys
sys.path.insert(0, '')

from ctypes import *
from nativelib cimport *
import nativelib as n


@CFUNCTYPE(UNCHECKED(n.NativeResult), n.NativeLibHdl)
def python_cfunctype_callback(libHdl) -> n.NativeResult:
    print("Do stuff in a CFUNCTYPE Python function here...")
    return n.NativeResult(n.RESULT_OK)

def python_regular_callback(libHdl: n.NativeLibHdl) -> n.NativeResult:
    print("Do stuff in a regular Python callback here...")
    return n.NativeResult(n.RESULT_OK)

cdef public NativeResult cythoncallbacks_getcallbacks(NativeLibHdl libHdl, CallbackParams params, NativeCallbacksHdl callbacksHdl):
    # How do I call python_cfunctype_callback or python_regular_callback from here?
    # Both ways below won't compile

    ###################################################
    
    # Errors out with
    # Cannot convert Python object to CallbackFunc* or
    # From https://stackoverflow.com/a/33485103/802203
    
    # callbacksHdl.callback = python_cfunctype_callback

    ###################################################
    
    # Errors out with
    # Storing unsafe C derivative of temporary Python reference
    # from https://stackoverflow.com/questions/2038839/python-ctypes-addressof-cfunctype
    
    # callbacktype = CFUNCTYPE(UNCHECKED(n.NativeResult), n.NativeLibHdl)
    # callbacksHdl.callback = callbacktype(python_cfunctype_callback)

    ####################################################################################

    return NativeResult(RESULT_NOT_IMPLEMENTED)

The sections above within the ###################### lines is what I'm trying to figure out. Ideally I'd prefer to use python_regular_callback if possible so I can load the appropriate module methods dynamically like I would with a dlopen and dlsym.

Thanks in advance, always glad to clarify or provide more information.

Ani
  • 10,826
  • 3
  • 27
  • 46
  • I think this might be a duplicate of https://stackoverflow.com/questions/49635105/ctypes-get-the-actual-address-of-a-c-function/49635841#49635841 but I'm not completely confident about it – DavidW Jan 22 '21 at 18:11
  • Interesting find, thanks! Although this is the opposite of what I need. I want to pass a Python function's address back to native code and call into it when needed from a native library. – Ani Jan 22 '21 at 18:23
  • Maybe I'm misunderstand, but I think that is what's happen in that answer: it's converting a ctypes wrapping of a Python function into a C function pointer, which you could then call from native code. – DavidW Jan 22 '21 at 19:05
  • Sorry, nothing from that linked answer works for me. I can't even copy that ctypedef with corresponding types from my code and have it work. I think a lot of that answer depends on the simple types required for the conversion. A large part of my question is because real-world C libraries don't pass basic types around, instead using enums, structs, opaque handles and typedef structs. Please do let me know if you're able to translate this in any way to my sample code on Github, linked above. – Ani Jan 24 '21 at 19:27
  • 1
    i think part of your problem is the definition of `CallbackFunc` in `NativeLib.pxd`. I believe it should be a C function pointer, for which the correct definition would be `ctypedef NativeResult (*CallbackFunc)(NativeLibHdl)`. After that, I'm afraid it's a larger example than I think I can get my head round right now – DavidW Jan 24 '21 at 20:10
  • No worries, thanks for your comments! I agree on your point about the pxd, so I'm trying to give it a go using Boost::Python instead of Cython since all my types are ctypes-wrapped correctly. Trying this way - https://mail.python.org/pipermail/cplusplus-sig/2012-September/016749.html – Ani Jan 24 '21 at 20:15
  • Thanks for your link! Even though it looked like it was the opposite of what I asked, it seems that's what I needed. See my working pyx here https://github.com/aniongithub/seamless-python-interop/blob/main/cythoncallbacks.pyx.in. – Ani Jan 26 '21 at 07:08
  • Glad it got there. I did think it was the right thing! – DavidW Jan 26 '21 at 07:31

0 Answers0