2

I have the following C function:

int c_sum(const int *a, int len) {

    int c = 0;
    for (int i=0; i<len; i++){
        c += a[i];
    }
    return c;
}

After creating the shared lib with the following:

cc -fPIC -shared -o c_sum.so c_sum.c

I run the following Python code in Jupyter notebook

import numpy as np
import time
from ctypes import *

lib = cdll.LoadLibrary("c_sum.so")
c_sum = lib.c_sum
c_sum.restype = c_int


a = np.random.randint(1,5,6)
print(a)

c = c_sum(c_void_p(a.ctypes.data), c_int(len(a)))
print(c)

It prints the following:

[1 4 3 1 2 2]
8

What might be the problem here? It prints the sum of the first 3 numbers. I tried random arrays of different lengths, it turns out that the code always sums the first half of the numbers in the array. Thanks.

DavidW
  • 29,336
  • 6
  • 55
  • 86
Sanyo Mn
  • 381
  • 1
  • 4
  • 12
  • 1
    1st thing is: [\[SO\]: C function called from Python via ctypes returns incorrect value (@CristiFati's answer)](https://stackoverflow.com/questions/58610333/c-function-called-from-python-via-ctypes-returns-incorrect-value/58611011#58611011). – CristiFati Jan 08 '22 at 14:42
  • Yes, I found that the problem is Python int is 4 bytes but C int is 2 bytes, when I change the type of the array from int to long it worked. But this seems to be an ad-hoc solution, I need to find a better way to handle this situtation. – Sanyo Mn Jan 08 '22 at 15:01
  • Your comment (at least the 1st sentence) couldn't be more wrong. All I mentioned in mine is that you should specify *argtypes* and *restype* for your function. – CristiFati Jan 08 '22 at 15:04
  • 1
    You need to change the int type of your array. Try `a.astype(np.int32)` (or `np.int16` if your C int is really 16-bit). – CJR Jan 08 '22 at 15:15
  • @CristiFati I'm not referring to your comment. All I'm referring to is: yes, it is about a problem related to the number of bytes allocated for integer types in C. – Sanyo Mn Jan 08 '22 at 15:57

1 Answers1

1

It's easier to explain in an answer rather than in comments. It has nothing to do with different int sizes. Yes, the types do differ, but not in the way OP thinks, as the int array is a NumPy (which is written in C and C++) one. Check [SO]: Maximum and minimum value of C types integers from Python for more details on this topic.

Listing [Python.Docs]: ctypes - A foreign function library for Python.
Also listing [NumPy]: C-Types Foreign Function Interface (numpy.ctypeslib) - convenience library for conversions between NumPy and CTypes.

dll00.c:

#if defined(_WIN32)
#  define DLL00_EXPORT_API __declspec(dllexport)
#else
#  define DLL00_EXPORT_API
#endif


#if defined(__cplusplus)
extern "C" {
#endif

DLL00_EXPORT_API int arrSum(const int *arr, int len);

#if defined(__cplusplus)
}
#endif


int arrSum(const int *arr, int len)
{
    int s = 0;
    for (int i = 0; i < len; ++i)
        s += arr[i];
    return s;
}

code00.py:

#!/usr/bin/env python

import ctypes as ct
import sys
import numpy as np


IntPtr = ct.POINTER(ct.c_int)

DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")


def main(*argv):
    dll00 = ct.CDLL(DLL_NAME)
    arrSum = dll00.arrSum
    arrSum.argtypes = (IntPtr, ct.c_int)
    arrSum.restype = ct.c_int

    a = np.random.randint(1, 5, 6)
    print(a, sum(a))

    datas = [
        np.ctypeslib.as_ctypes(a),  # Using ctypeslib
        a.ctypes.data_as(IntPtr),  # Using manual conversion
        #IntPtr(ct.c_long(a.ctypes.data)),  # Original way - wrong
    ]
    for data in datas:
        res = arrSum(data, len(a))
        print("\n{0:s} returned: {1:d}".format(arrSum.__name__, res))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.")
    sys.exit(rc)

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q070633293]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 >nul

[prompt]> dir /b
code00.py
dll00.c

[prompt]> cl /nologo /MD /DDLL dll00.c  /link /NOLOGO /DLL /OUT:dll00.dll
dll00.c
   Creating library dll00.lib and object dll00.exp

[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts\python.exe" code00.py
Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] 064bit on win32

[2 2 2 3 4 4] 17

arrSum returned: 17

arrSum returned: 17

Done.
CristiFati
  • 38,250
  • 9
  • 50
  • 87
  • If you're setting argtype for something that gets a numpy array as a pointer, I would suggest using numpy.ctypeslib.ndpointer instead of a generic pointer type. You can check that a bunch of attributes and flags on the python object are correct. – CJR Jan 08 '22 at 16:49
  • 1
    @CJR: That's true, thanks for the suggestion, it could be `arrSum.argtypes = (np.ctypeslib.ndpointer(dtype=np.int32, ndim=1, flags='C_CONTIGUOUS'), ct.c_int)`, and the call, simply: `res = arrSum(a, len(a))`. – CristiFati Jan 08 '22 at 17:14