1

I'm writing Python bindings for a C library that heavily uses callbacks. For convenience in Python, I'm searching for a way to pass a Python object as void * to the C function, and once it gives the pointer back to me in the Python callback, convert it back to the Python object.

First, here is an example C file:

test1.c

void call_callback(void (*callback)(void *), void *user_data)
{
    callback(user_data);
}

Compile this to a shared library named libtest1.so. Here is an example Python file:

test1.py

# Create bindings (in reality done in a module)
from ctypes import *

cdll.LoadLibrary("libtest1.so")
test1 = CDLL("libtest1.so")

test1.call_callback.argtypes = [CFUNCTYPE(None, c_void_p), c_void_p]
test1.call_callback.restype = None
def call_callback(func, data):
    # Note: converting Python function to be callable from C
    callback = CFUNCTYPE(None, c_void_p)(func)
    test1.call_callback(callback, data)

# Testing:
def callback1(data):
    number = cast(data, POINTER(c_int))
    print('Callback 1 got:', number[0])

data1 = c_int(10)
call_callback(callback1, byref(data1))

So, far everything works ok. Data to be sent can be created with ctypes.c_int for example and sent by ctypes.byref, and later recovered by a ctypes.cast to the appropriate type (ctypes.POINTER(ctypes.c_int)).

My question is the following. Is it possible for me to give any normal Python object as a ctypes.c_void_p when calling a C function? If so, how? Also, how can I recover the Python object when I get that pointer back in a Python callback?

Imagine the following code:

def callback2(data):
    list = c_void_p_was_really_a_python_list(data)
    print('callback 2 got:', list)

data2 = [1, 2, 3]
call_callback(callback2, convert_to_c_void_p(data2))

I am essentially looking for the convert_to_c_void_p and c_void_p_was_really_a_python_X functions.

A solution that works both with Python 2 (>= 2.7) and Python 3 would be great, but I would be interested in version specific answers as well.

Shahbaz
  • 46,337
  • 19
  • 116
  • 182
  • 2
    You can use `py_object` if you must. If the C function's `argtypes` uses `c_void_p`, then use `convert_to_c_void_p = lambda obj: c_void_p.from_buffer(py_object(obj))`. – Eryk Sun Mar 30 '15 at 20:38
  • 1
    In the callback use `obj = cast(data, py_object).value`. Be careful about long-term references. If the C function calls the callback immediately before returning this should be fine. Otherwise the `data` address passed in may reference a deallocated object and at best you'll segfault. To prevent this you'd have to keep a manual proxy reference in Python to account for the reference held by the C library. This is needlessly complicated considering you don't need this `data` parameter at all. This is Python. You can use bound methods and closures. – Eryk Sun Mar 30 '15 at 20:44
  • @eryksun, awesome! You are absolutely right, and in C the `data` parameter is actually to have closures, which python already does for you. However, your solution to converting to `py_object` and back is the correct solution to the exact question I asked (I tested it as well). Please consider writing up an answer so I can accept it. – Shahbaz Mar 31 '15 at 09:43
  • I found [this question](http://stackoverflow.com/q/2469975/912144) by accident searching for something else. This question is a duplicate of that one. – Shahbaz Mar 31 '15 at 16:19
  • 1
    Avoid using `cdata = cast(pointer(py_object(obj)), c_void_p)`, as shown in the comments of that question. That requires keeping a reference to `cdata`, which references the `py_object` instance, which references `obj`. It wastes creating a pointer and doing a FFI call for the `cast` (it uses the function pointer `ctypes._cast` based on `_ctypes._cast_addr`), plus it complicates the `cast` back to `obj` in the callback. – Eryk Sun Mar 31 '15 at 19:39
  • @eryksun, realizing bound methods and closures work, I would certainly not use this method. Anyway, please vote to close this question as the duplicate I found. – Shahbaz Apr 01 '15 at 08:38
  • Related question needing an answer: https://stackoverflow.com/questions/60219216/can-i-pass-a-python-self-object-to-a-c-function-as-c-void-p-and-cast-to-origin – NKatUT Feb 14 '20 at 12:51

0 Answers0