13

The C function myfunc operates on a larger chunk of data. The results are returned in chunks to a callback function:

int myfunc(const char *data, int (*callback)(char *result, void *userdata), void *userdata);

Using ctypes, it's no big deal to call myfunc from Python code, and to have the results being returned to a Python callback function. This callback work fine.

myfunc = mylib.myfunc
myfunc.restype = c_int
myfuncFUNCTYPE = CFUNCTYPE(STRING, c_void_p)
myfunc.argtypes = [POINTER(c_char), callbackFUNCTYPE, c_void_p]

def mycb(result, userdata):
    print result
    return True

input="A large chunk of data."
myfunc(input, myfuncFUNCTYPE(mycb), 0)

But, is there any way to give a Python object (say a list) as userdata to the callback function? In order to store away the result chunks, I'd like to do e.g.:

def mycb(result, userdata):
    userdata.append(result)

userdata=[]

But I have no idea how to cast the Python list to a c_void_p, so that it can be used in the call to myfunc.

My current workaround is to implement a linked list as a ctypes structure, which is quite cumbersome.

flight
  • 1,738
  • 2
  • 14
  • 14

1 Answers1

4

I guess you could use the Python C API to do that... maybe you could use a PyObject pointer.

edit: As the op pointed out in the comments, there's already a py_object type readily available in ctypes, so the solution is to create first a ctypes.py_object object from the python list and then casting it to c_void_p to pass it as an argument to the C function (I think this step might be unnecessary as a parameter typed as void* should accept any pointer, and it would be faster to pass just a byref). In the callback, the reverse steps are done (casting from the void pointer to a pointer to py_object and then getting the value of the contents).

A workaround could be to use a closure for your callback function so it already knows in which list it has to append the items...

myfunc = mylib.myfunc
myfunc.restype = c_int
myfuncFUNCTYPE = CFUNCTYPE(STRING)
myfunc.argtypes = [POINTER(c_char), callbackFUNCTYPE]


def mycb(result, userdata):
    userdata.append(result)

input="A large chunk of data."
userdata = []
myfunc(input, myfuncFUNCTYPE(lambda x: mycb(x, userdata)))
Alexander
  • 23,432
  • 11
  • 63
  • 73
fortran
  • 74,053
  • 25
  • 135
  • 175
  • Well, the nice thing about ctypes is that everything can be done in plain Python. So the Python C API seems to be a step back. Or am I missing something? – flight Mar 18 '10 at 13:27
  • I meant the option could be creating a ctypes proxy of a C-API PyObject struct, that should be the internal representation of any Python object and passing it around... – fortran Mar 18 '10 at 13:30
  • Strange. Your workaround with a closure works fine on Linux, but the same program crashes on Windows. – flight Mar 19 '10 at 10:13
  • Have you noticed he change in the signature of the callback function and changed that accordingly in your C files? – fortran Mar 19 '10 at 10:53
  • Nope. That's certainly the problem. I wondered why it worked without the change on Linux. – flight Mar 19 '10 at 10:58
  • 10
    But I found the solution I was looking for: `cUserdata=ctypes.cast(pointer(ctypes.py_object(pyUserdata)), c_void_p)` and in the callback: `pyUserdata=ctypes.cast(cUserdata,POINTER(ctypes.py_object)).contents.value`. I guess that's what had in mind with the Python C API? – flight Mar 19 '10 at 11:04
  • 1
    About the first question, it probably has to do with the calling conventions, by examining what happened I'd say the parameter in Linux was expected in a register and in Windows in the stack (and as it was missing, the stack was corrupted at some point). Is your Linux 64 bit and your Windows 32 bit? – fortran Mar 19 '10 at 11:36
  • 1
    About the second, that's more or less what I was suggesting, using the C internal representation of the Python object... Luckily enough, ctypes already had it implemented and you didn't have to write your own wrapper and some C code to get the actual pointer. – fortran Mar 19 '10 at 11:39
  • 1
    Both 32 bit. But in fact, the Windows library uses WINFUNCTYPE, while the Linux version uses CFUNCTYPE. – flight Mar 19 '10 at 11:42
  • Would you mind to update your answer with the py_object? Again, thanks for the hint! – flight Mar 19 '10 at 11:44