Pass str as an int array to a Python C extended function (extended using SWIG)

Question

How can I pass a str value (containing 3000 {'0', '1'} bytes) obtained using python code as an argument to a python c extended function (extended using SWIG) which requires int * (fixed length int array) as an input argument? My code is such:

int *exposekey(int *bits) {
    int a[1000];
    for (int j=2000; j < 3000; j++) {
        a[j - 2000] = bits[j];
    }
    return a;
}

What I've tried was to use ctypes (see below code):

import ctypes
ldpc = ctypes.cdll.LoadLibrary('./_ldpc.so')
arr = (ctypes.c_int * 3072)(<mentioned below>)
ldpc.exposekey(arr)

with 3072 {0, 1} entered in the position. Python returns syntax error : more than 255 arguments. This still doesn't help me to pass assigned str value instead of the initialized ctypes int array.

Other suggestion included using SWIG typemaps but how would that work for converting a str into int * ? Thanks in advance.

Returning local arrays from functions is a tricky thing since they reside on the stack, and get destroyed when going out of scope, and thus later on when the returned address will be dereferenced, you'll most likely get a _segfault_. Either make it `static`, or dynamically allocate it, or add it as a 2nd (output) argument to your function. I'd go with #3. — CristiFati, Nov 14 '17 at 14:29

CristiFati · Accepted Answer · 2023-05-15T09:38:41.313

Regarding my comment, here are some more details about returning arrays from functions: [SO]: Returning an array using C. In short, ways (each with its pros and cons) to handle this:

Make the returned variable static
Dynamically allocate it (using malloc (family) or new)
Turn it into an additional argument for the function (and delegate management to the caller)

Getting that piece of C (or C++) code to run within the Python interpreter is possible in 2 ways:

[Python.Docs]: Extending Python with C or C++ - which creates an extension (C written) Python module
- That's typically done using [Python.Docs]: Python/C API Reference Manual. Since the boiler plate code is quite large, there are 3^rd-party tools (wrappers) which make things easier (offering a simpler interface for generating the module):
The other way around, leaving the code in a standard .dll (.so) which can be accessed via [Python.Docs]: ctypes - A foreign function library for Python

Since they both are doing the same thing, mixing them together makes no sense. So, pick the one that best fits your needs (I want to mention that running your code didn't raise the error you got).

I prepared examples for the methods. They work using Python 3.

1. SWIG

swig_demo.h:

#define OUT_BUF_SIZE 1000


char* exposekey(const char *bIn);

swig_demo.c:

#include <malloc.h>
#include <stdio.h>
#include "swig_demo.h"


char* exposekey(const char *bIn)
{
    char *ret = (char*)malloc(sizeof(char) * 20000000);
    if (ret == NULL) {
        return NULL;
    }
    printf("Message from C code...\n");
    for (int j = 0; j < OUT_BUF_SIZE; ++j) {
        ret[j] = bIn[j + 2000];
    }
    return ret;
}

swig_demo.i:

%module swig_demo
%{
#define SWIG_PYTHON_STRICT_BYTE_CHAR
#include "swig_demo.h"
%}

%newobject exposekey;
%include "swig_demo.h"

test_mod.py (shared by all methods that create an extension module):

#!/usr/bin/env python

import sys 

import data

try:
    from _capi_demo import exposekey as capi_exposekey
except:
    capi_exposekey = None

try:
    from swig_demo import exposekey as swig_exposekey
except:
    swig_exposekey = None

try:
    from _cython_demo import exposekey as cython_exposekey
except:
    cython_exposekey = None


def main(*argv):
    eks = (
        capi_exposekey,
        swig_exposekey,
        cython_exposekey,
    )
    buf = data.gen_buf()
    for ek in eks:
        if ek is None:
            continue
        print("Running function: {:s} from {:s}".format(ek.__name__, ek.__module__))
        out = ek(buf)
        print("C function returned ({:d}): {:}".format(len(out), out))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

data.py (shared by all methods):

def gen_buf(pad=2000, length=1000):
    _buf = b"010011000110101110101110101010010111011101101010101"
    d, m = divmod(length, len(_buf))
    return b"\x00" * pad + _buf * d + _buf[:m]

Makefile.mak (same):

!include <NtWin32.mak>


.PHONY: capi swig cython ctypes

PYTHON_DIR=c:\Install\pc064\Python\Python\03.10
PYTHON_DIR_INC=$(PYTHON_DIR)\include
PYTHON_DIR_LIB=$(PYTHON_DIR)\libs

RM=del /f /q

SRC_DIR=src
CAPI_DEMO=capi_demo
SWIG_DEMO=swig_demo
CYTHON_DEMO=cython_demo
CTYPES_DEMO=ctypes_demo
SWIG_DEMO_WRAP=$(SWIG_DEMO)_wrap
CYTHON_DEMO_SRC=$(CYTHON_DEMO).c

SWIG_DEMO_SRC=$(SRC_DIR)\$(SWIG_DEMO).i
SWIG_DEMO_WRAP_SRC=$(SWIG_DEMO_WRAP).c

CYTHON_DEMO_PYX=$(SRC_DIR)\$(CYTHON_DEMO).pyx
CYTHON_DEMO_SRC=$(CYTHON_DEMO).c

CAPI_DEMO_OBJ=$(CAPI_DEMO).obj

SWIG_DEMO_OBJ=$(SWIG_DEMO).obj
SWIG_DEMO_WRAP_OBJ=$(SWIG_DEMO_WRAP).obj

CAPI_OBJS=$(CAPI_DEMO_OBJ) $(SWIG_DEMO_OBJ)

SWIG_OBJS=$(SWIG_DEMO_OBJ) $(SWIG_DEMO_WRAP_OBJ)

CYTHON_DEMO_OBJ=$(CYTHON_DEMO).obj

CYTHON_OBJS=$(CYTHON_DEMO_OBJ) $(SWIG_DEMO_OBJ)

CTYPES_DEMO_OBJ=$(CTYPES_DEMO).obj

CAPI_TARGET=_$(CAPI_DEMO).pyd
SWIG_TARGET=_$(SWIG_DEMO).pyd
CTYHON_TARGET=_$(CYTHON_DEMO).pyd
CTYPES_TARGET=$(CTYPES_DEMO).dll

#cflags=$(cflags) /UDEBUG /U_DEBUG /DNDEBUG


all:

clean:
    -@$(RM) *.dll *.pyd *.lib *.exp *.pdb *.obj $(SWIG_DEMO).py $(SWIG_DEMO_WRAP_SRC) $(CYTHON_DEMO_SRC)

.c.obj:
    $(cc) $(cdebug) $(cflags) $(cvarsdll) /I"$(SRC_DIR)" /I"$(PYTHON_DIR_INC)" /c /Fo$@ $<

{$(SRC_DIR)}.c.obj:
    $(cc) $(cdebug) $(cflags) $(cvarsdll) /I"$(PYTHON_DIR_INC)" /c /Fo$@ $<

$(SWIG_DEMO_WRAP_SRC): $(SWIG_DEMO_SRC)
    swig -python -py3 -o $@ $?

$(CYTHON_DEMO_SRC): $(CYTHON_DEMO_PYX)
    cython.exe -3 --module-name _$(CYTHON_DEMO) -o $(CYTHON_DEMO_SRC) $?

$(CAPI_TARGET): $(CAPI_OBJS)
    $(link) $(ldebug) $(dlllflags) /LIBPATH:"$(PYTHON_DIR_LIB)" $(conlibsdll) /OUT:$@ $?

$(SWIG_TARGET): $(SWIG_OBJS)
    $(link) $(ldebug) $(dlllflags) /LIBPATH:"$(PYTHON_DIR_LIB)" $(conlibsdll) /OUT:$@ $?

$(CTYHON_TARGET): $(CYTHON_OBJS)
    $(link) $(ldebug) $(dlllflags) /LIBPATH:"$(PYTHON_DIR_LIB)" $(conlibsdll) /OUT:$@ $?

$(CTYPES_TARGET): $(CTYPES_DEMO_OBJ)
    $(link) $(ldebug) $(dlllflags) $(conlibsdll) /OUT:$@ $?


capi: $(CAPI_TARGET)
swig: $(SWIG_TARGET)
cython: $(CTYHON_TARGET)
ctypes: $(CTYPES_TARGET)

Notes:

Based on comments, I changed the types in the function from int* to char*, because it's 4 times more compact (although it's still ~700% inefficient since 7 bits of each char are ignored versus only one of them being used; that can be fixed, but requires bitwise processing)
I also modified the index range (without changing functionality), because it makes more sense to work with low index values and add something to them in one place, instead of a high index values and subtract (the same) something in another place
printf it's just dummy, to show that the C code gets executed
When dealing with such arrays, it's recommended to pass their dimensions as well, to avoid out of bounds errors. Also, error handling is an important aspect
I'm allocating the array and return it (the 2^nd option from the beginning)
The .i file is a standard SWIG interface file
- Defines the module, and its exports (via %include)
- One thing that is worth mentioning is the %newobject directive that deallocates the pointer returned by exposekey to avoid memory leaks
The .h file just contains the function declaration, in order to be included by the .i file (it's not mandatory, but things are more elegant this way)
In Python code (data.py), I generate an input buffer 3000 (padded with 2000 NUL bytes at the beginning) bytes long and the output is 1000 (hardcoded values based on your example didn't make them configurable, to keep the code simple)
Check [SO]: Visual Studio NMake build fails with: fatal error U1052: file 'win32.mak' not found (@CristiFati's answer) for details regarding Win builds

Output (on Win. I'll be reusing the console):

[cfati@CFATI-5510-0:e:\Work\Dev\StackExchange\StackOverflow\q047276327]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul

[prompt]> set _INCLUDE=%INCLUDE%

[prompt]> set INCLUDE=%_INCLUDE%;e:\Work\Dev\GitHub\CristiFati\MSSDKFiles\src\Include

[prompt]> set _PATH=%PATH%

[prompt]> set PATH=%_PATH%;e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts;f:\Install\pc032\SWIG\SWIGWin\4.0.1

[prompt]>
[prompt]> tree /a /f
Folder PATH listing for volume SSD0-WORK
Volume serial number is AE9E-72AC
E:.
|   data.py
|   Makefile.mak
|   test_ctypes.py
|   test_mod.py
|
\---src
        capi_demo.c
        ctypes_demo.c
        cython_demo.pyx
        swig_demo.c
        swig_demo.h
        swig_demo.i


[prompt]>
[prompt]> nmake /f Makefile.mak NODEBUG=1 clean swig

Microsoft (R) Program Maintenance Utility Version 14.29.30148.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Could Not Find e:\Work\Dev\StackExchange\StackOverflow\q047276327\*.dll
        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"c:\Install\pc064\Python\Python\03.10\include" /c /Foswig_demo.obj src\swig_demo.c
swig_demo.c
        swig -python -py3 -o swig_demo_wrap.c src\swig_demo.i
        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"src" /I"c:\Install\pc064\Python\Python\03.10\include" /c /Foswig_demo_wrap.obj swig_demo_wrap.c
swig_demo_wrap.c
c:\Install\pc064\Python\Python\03.10\include\cpython/pytime.h(120): warning C4115: 'timeval': named type definition in parentheses
swig_demo_wrap.c(1437): warning C4100: 'args': unreferenced formal parameter
swig_demo_wrap.c(1566): warning C4100: 'args': unreferenced formal parameter
swig_demo_wrap.c(1578): warning C4100: 'args': unreferenced formal parameter
swig_demo_wrap.c(1586): warning C4100: 'args': unreferenced formal parameter
swig_demo_wrap.c(2361): warning C4100: 'clientdata': unreferenced formal parameter
swig_demo_wrap.c(2537): warning C4100: 'argnum': unreferenced formal parameter
swig_demo_wrap.c(2837): warning C4100: 'self': unreferenced formal parameter
swig_demo_wrap.c(3157): warning C4100: 'v': unreferenced formal parameter
swig_demo_wrap.c(3448): warning C4100: 'self': unreferenced formal parameter
swig_demo_wrap.c(3467): warning C4100: 'self': unreferenced formal parameter
        link /RELEASE  /INCREMENTAL:NO /NOLOGO -entry:_DllMainCRTStartup -dll /LIBPATH:"c:\Install\pc064\Python\Python\03.10\libs" kernel32.lib  ws2_32.lib mswsock.lib advapi32.lib /OUT:_swig_demo.pyd swig_demo.obj swig_demo_wrap.obj
   Creating library _swig_demo.lib and object _swig_demo.exp

[prompt]>
[prompt]> dir /b
data.py
Makefile.mak
src
swig_demo.obj
swig_demo.py
swig_demo_wrap.c
swig_demo_wrap.obj
test_ctypes.py
test_mod.py
_swig_demo.exp
_swig_demo.lib
_swig_demo.pyd

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" test_mod.py
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

Running function: exposekey from swig_demo
Message from C code...
C function returned (1000): b'0100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100'

Done.

2. Cython

cython_demo.pyx:

from libc.stdlib cimport free


cdef extern from "swig_demo.h":
    char* _exposekey "exposekey"(const char *bIn)


def exposekey(bIn):
    cdef char *b = _exposekey(bIn)
    ret = bytes(b)
    free(b)
    return ret

Notes:

I added this part as a personal exercise
Requires swig_demo.h and swig_demo.c (the actual function)

Output:

[prompt]> nmake /f Makefile.mak NODEBUG=1 clean cython

Microsoft (R) Program Maintenance Utility Version 14.29.30148.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        cython.exe -3 --module-name _cython_demo src\cython_demo.pyx
        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"c:\Install\pc064\Python\Python\03.10\include" /c /Focython_demo.obj src\cython_demo.c
cython_demo.c
c:\Install\pc064\Python\Python\03.10\include\cpython/pytime.h(120): warning C4115: 'timeval': named type definition in parentheses
src\cython_demo.c(1264): warning C4100: '__pyx_self': unreferenced formal parameter
src\cython_demo.c(1564): warning C4100: 'def': unreferenced formal parameter
src\cython_demo.c(2289): warning C4127: conditional expression is constant

:: ------- @TODO - cfati: Truncated output (same warning) -------

src\cython_demo.c(3116): warning C4127: conditional expression is constant
        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"c:\Install\pc064\Python\Python\03.10\include" /c /Foswig_demo.obj src\swig_demo.c
swig_demo.c
        link /RELEASE  /INCREMENTAL:NO /NOLOGO -entry:_DllMainCRTStartup -dll /LIBPATH:"c:\Install\pc064\Python\Python\03.10\libs" kernel32.lib  ws2_32.lib mswsock.lib advapi32.lib /OUT:_cython_demo.pyd cython_demo.obj swig_demo.obj
   Creating library _cython_demo.lib and object _cython_demo.exp

[prompt]>
[prompt]> dir /b
cython_demo.c
cython_demo.obj
data.py
Makefile.mak
src
swig_demo.obj
test_ctypes.py
test_mod.py
_cython_demo.exp
_cython_demo.lib
_cython_demo.pyd
__pycache__

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" test_mod.py
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

Running function: exposekey from _cython_demo
Message from C code...
C function returned (1000): b'0100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100'

Done.

3. Plain Python C API

capi_demo.c:

#define PY_SSIZE_T_CLEAN
#include "Python.h"
#include "swig_demo.h"

#define MOD_NAME "_capi_demo"


static PyObject* PyExposekey(PyObject *self, PyObject *args)
{
    PyObject *bitsInArg = NULL, *ret = NULL;
    char *bitsOut = NULL;
    if (!PyArg_ParseTuple(args, "S", &bitsInArg))
        return NULL;
    bitsOut = exposekey(PyBytes_AsString(bitsInArg));
    ret = PyBytes_FromStringAndSize(bitsOut, OUT_BUF_SIZE);
    free(bitsOut);
    return ret;
}


static PyMethodDef moduleMethods[] = {
    {"exposekey", (PyCFunction)PyExposekey, METH_VARARGS, NULL},
    {NULL, NULL, 0, NULL},  // Sentinel
};

static struct PyModuleDef moduleDef = {
    PyModuleDef_HEAD_INIT, MOD_NAME, NULL, -1, moduleMethods
};


PyMODINIT_FUNC PyInit__capi_demo(void)
{
    return PyModule_Create(&moduleDef);
}

Notes:

Also a personal exercise
This is what SWIG / Cython do, but "manually"
Error handling may be improved
Also requires the actual function

Output:

[prompt]> nmake /f Makefile.mak NODEBUG=1 clean capi

Microsoft (R) Program Maintenance Utility Version 14.29.30148.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"c:\Install\pc064\Python\Python\03.10\include" /c /Focapi_demo.obj src\capi_demo.c
capi_demo.c
c:\Install\pc064\Python\Python\03.10\include\cpython/pytime.h(120): warning C4115: 'timeval': named type definition in parentheses
src\capi_demo.c(8): warning C4100: 'self': unreferenced formal parameter
        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"c:\Install\pc064\Python\Python\03.10\include" /c /Foswig_demo.obj src\swig_demo.c
swig_demo.c
        link /RELEASE  /INCREMENTAL:NO /NOLOGO -entry:_DllMainCRTStartup -dll /LIBPATH:"c:\Install\pc064\Python\Python\03.10\libs" kernel32.lib  ws2_32.lib mswsock.lib advapi32.lib /OUT:_capi_demo.pyd capi_demo.obj swig_demo.obj
   Creating library _capi_demo.lib and object _capi_demo.exp

[prompt]>
[prompt]> dir /b
capi_demo.obj
data.py
Makefile.mak
src
swig_demo.obj
test_ctypes.py
test_mod.py
_capi_demo.exp
_capi_demo.lib
_capi_demo.pyd
__pycache__

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" test_mod.py
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

Running function: exposekey from _capi_demo
Message from C code...
C function returned (1000): b'0100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100'

Done.

4. CTypes

ctypes_demo.c:

#include <stdio.h>

#if defined(_WIN32)
#  define CTYPES_DEMO_EXPORT_API __declspec(dllexport)
#else
#  define CTYPES_DEMO_EXPORT_API
#endif

#define OUT_BUF_SIZE 1000


int CTYPES_DEMO_EXPORT_API exposekey(const char *bIn, char *bOut)
{
    int ret = 0;
    printf("Message from C code...\n");
    for (int j = 0; j < OUT_BUF_SIZE; ++j) {
        bOut[j] = bIn[j + 2000];
        ++ret;
    }
    return ret;
}

test_ctypes.py:

#!/usr/bin/env python

import ctypes as cts
import sys

from data import gen_buf


OUT_BUF_SIZE = 1000
DLL_NAME = "./ctypes_demo.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")


def main(*argv):
    dll = cts.CDLL(DLL_NAME)
    exposekey = dll.exposekey
    exposekey.argtypes = (cts.c_char_p, cts.c_char_p)
    exposekey.restype = cts.c_int

    bin = cts.create_string_buffer(gen_buf())
    bout = cts.create_string_buffer(OUT_BUF_SIZE)
    print("Before ({:d}): {:} ... {:}".format(len(bout), bout.raw[:100], bout.raw[-100:]))
    ret = exposekey(bin, bout)
    print("After ({:d}): {:}".format(len(bout), bout.raw))
    print("Return code: {:d}".format(ret))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

Notes:

This is what you started with
It's one of the ways of doing things using CTypes
I took a and turned into the 2^nd argument (bOut). I think this is best because it's caller responsibility to allocate and deallocate the array (the 3^rd option from the beginning). The alternative would be to export another function that deallocates whatever exposekey returns
Return value is the number of bits set (obviously, 1000 in this case) but it's just an example
Check [SO]: C function called from Python via ctypes returns incorrect value (@CristiFati's answer) for a common pitfall when working with CTypes (calling functions)

Output:

[prompt]> nmake /f Makefile.mak NODEBUG=1 clean ctypes

Microsoft (R) Program Maintenance Utility Version 14.29.30148.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        cl -Ox -DNDEBUG -c -DCRTAPI1=_cdecl -DCRTAPI2=_cdecl -nologo -GS -D_AMD64_=1 -DWIN64 -D_WIN64  -DWIN32 -D_WIN32 -W4 -D_WINNT -D_WIN32_WINNT=0x0600 -DNTDDI_VERSION=0x06000000 -D_WIN32_IE=0x0700 -DWINVER=0x0600  -D_MT -D_DLL -MD /I"c:\Install\pc064\Python\Python\03.10\include" /c /Foctypes_demo.obj src\ctypes_demo.c
ctypes_demo.c
        link /RELEASE  /INCREMENTAL:NO /NOLOGO -entry:_DllMainCRTStartup -dll kernel32.lib  ws2_32.lib mswsock.lib advapi32.lib /OUT:ctypes_demo.dll ctypes_demo.obj
   Creating library ctypes_demo.lib and object ctypes_demo.exp

[prompt]>
[prompt]> dir /b
ctypes_demo.dll
ctypes_demo.exp
ctypes_demo.lib
ctypes_demo.obj
data.py
Makefile.mak
src
test_ctypes.py
test_mod.py
__pycache__

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" test_ctypes.py
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

Before (1000): b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' ... b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
Message from C code...
After (1000): b'0100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100101110111011010101010100110001101011101011101010100'
Return code: 1000

Done.

Thank you soo much . this works perfectly. But like you said i'am trying to apply thins logic to complex functions/projects , hence the use of SWIG. — 4am, Nov 15 '17 at 06:07
In the above code you are defining the input array to be passed, if have an input string of 0's and 1's (type= python str) how do i pass that as input to this ctypes func? Thanks again. — 4am, Nov 15 '17 at 06:24
If you want to pass a string, you need to change the function (so its `bits` argument is a `char*`), and also from _Python_, its `argtypes`. It's definitely better to store a **0/1** value (1bit) in a `char` (8bits) than in an `int` (32bits), however it's still 700% inefficient from the storage's *PoV* (as the `char` 7 other bits aren't used). While on this subject, if changing `bits` wouldn't it be a good idea to also change `out` (to `char*`)? Or you could go the easy way (initialize `bits` like this): `for i, char in enumerate(your_string):` `bits[2000 + i] = ord(char) - ord("0")`. — CristiFati, Nov 15 '17 at 08:26
tried what you suggested passing to char * and did the bit conversion at the c end using atoi(), instead of ord() at python end. Thank you — 4am, Nov 17 '17 at 12:04

Pass str as an int array to a Python C extended function (extended using SWIG)

1 Answers1

1. SWIG

2. Cython

3. Plain Python C API

4. CTypes

Linked