Why am I getting negative ptr when I use Ctypes with dynamic fn name?

Question

I have a below python script which works,

from ctypes import c_uint32, c_uint8, c_void_p, c_char_p
lib = ctypes.cdll.LoadLibrary(dll_dir_name + "my_dll.dll")
lib.my_func.argtype = c_char_p
lib.my_func.restype = c_void_p
ptr = lib.my_func("Hello".encode('utf-8'))
ptr
# ptr is positive value and below command works
raw_json = ctypes.cast(ptr, c_char_p).value.decode('utf-8')

However, When I use fn name as a variable, python is crashing,

from ctypes import c_uint32, c_uint8, c_void_p, c_char_p
lib = ctypes.cdll.LoadLibrary(dll_dir_name + "my_dll.dll")
fn_name = "my_func"
lib[fn_name].argtype = c_char_p
lib[fn_name].restype = c_void_p
ptr = lib[fn_name]("Hello".encode('utf-8'))
ptr
# ptr is negative value and command crashes python
raw_json = ctypes.cast(ptr, c_char_p).value.decode('utf-8')

[\[SO\]: C function called from Python via ctypes returns incorrect value (@CristiFati's answer)](https://stackoverflow.com/questions/58610333/c-function-called-from-python-via-ctypes-returns-incorrect-value/58611011#58611011) — CristiFati, Jul 21 '20 at 19:53
`.argtypes` not `.argtype` but there is more than that error going on... — Mark Tolonen, Jul 22 '20 at 00:27

Mark Tolonen · Accepted Answer · 2020-07-22T00:23:57.510

The reason is each time you use lib[fn_name] it returns a different _FuncPtr instance. I'll use the following C code that cannot work with ctypes without correct .argtypes (plural) and .restype (singular):

test.c:

__declspec(dllexport)             // Needed for Windows...
API double func(double a) {
    return a * 1.5;
}

Example:

>>> from ctypes import *
>>> dll = CDLL('test')
>>> 'func' in dir(dll)     # attribute doesn't exist yet
False
>>> dll.func               # attribute is created caching a single _FuncPtr instance.
<_FuncPtr object at 0x0000018455E00798>
>>> 'func' in dir(dll)
True
>>> dll.func               # fetches the same instance (same address)
<_FuncPtr object at 0x0000018455E00798>  
>>> dll['func']            # Creates a new _FuncPtr instance each time
<_FuncPtr object at 0x0000018455E00868>
>>> dll['func']            # different
<_FuncPtr object at 0x0000018455E00938>
>>> dll['func']            # different
<_FuncPtr object at 0x0000018455E00A08>

So setting attributes use a different dll['func'] instance doesn't set them correctly. The following code fails:

from ctypes import *

dll = CDLL('test')

f = 'func'
dll[f].argtypes = c_double,  # creates an instance, sets the attribute, but doesn't save instance
dll[f].restype = c_double    # ditto
print(dll[f](2.5))           # calls function with no attributes set.

Output:

Traceback (most recent call last):
  File "C:\test.py", line 8, in <module>
    print(dll[f](2.5))
ctypes.ArgumentError: argument 1: <class 'TypeError'>: Don't know how to convert parameter 1

The default when .argtypes is not set is to assume c_int and Python doesn't know how to convert a Python float to a c_int.

The fix is to lookup the function once and use the same instance:

from ctypes import *

dll = CDLL('test')

f = dll['func']
f.argtypes = c_double,
f.restype = c_double
print(f(2.5))

Output:

3.75

Another option is to use getattr() which looks up and caches the named attribute, so future calls return the same address:

>>> from ctypes import *
>>> dll = CDLL('test')
>>> 'func' in dir(dll)   # attribute doesn't exist
False
>>> f = 'func'
>>> getattr(dll,f)  # look up attribute by name
<_FuncPtr object at 0x0000028171EE0798>
>>> 'func' in dir(dll)   # attribute exists
True
>>> getattr(dll,f)       # looks up cached attribute.
<_FuncPtr object at 0x0000028171EE0798>

So this will also work:

from ctypes import *

dll = CDLL('test')

f = 'func'
getattr(dll,f).argtypes = c_double,
getattr(dll,f).restype = c_double
print(getattr(dll,f)(2.5))

Output:

3.75

But still cleaner to just use f = getattr(dll,'func') once and use f instead of multiple lookups.

Why am I getting negative ptr when I use Ctypes with dynamic fn name?

1 Answers1