0

I want to call a C function from Python, which should create and initialize a struct. I want this struct converted to a Python object as the return value in Python.

Flexo
  • 87,323
  • 22
  • 191
  • 272
user2804197
  • 354
  • 5
  • 13
  • As far as I am aware, you cannot return the (bare) struct itself to Python. Python functions must return Python objects. – John Bollinger Mar 28 '17 at 14:14
  • I don't want the bare struct but a Python object, which is what the `%typemap(argout)` should provide – user2804197 Mar 28 '17 at 14:19
  • Correction: the example C code does not return anything at all, neither via its parameter nor via a return value. It just modifies its parameter, which is local to the function, not the object to which it points. – John Bollinger Mar 28 '17 at 14:22
  • it's good that you want to return a Python object, but that contradicts what you actually wrote: "I want this struct as the return value in Python." – John Bollinger Mar 28 '17 at 14:24
  • John Bollinger: I think I fixed my code. The result is the same. I also rephrased the "I want this struct as the return value in Python" sentence. – user2804197 Mar 28 '17 at 14:35
  • Yes, you seem to have arrived at a version of the `create_struct()` function that is not inherently erroneous. I am uncertain whether it is any longer a good example for the problem you are asking about, or indeed whether it ever was in the first place. But we can go on from here. – John Bollinger Mar 28 '17 at 14:37
  • At this point it's worth talking about the typemaps -- at least to observe that those you've presented do nothing for you. You are mapping the type of a *member* of `struct Foo`, but typemaps are relevant only to the types of functions' parameters and return values. The type you're mapping is not used in either capacity. As the code is now written, you need a mapping for `struct Foo**`. – John Bollinger Mar 28 '17 at 14:38
  • I also observe that although your function is now useful, you have created a problem for yourself via the dynamic allocation: you'll have a memory leak unless you also arrange to *free* the allocated memory when you're done with it. – John Bollinger Mar 28 '17 at 14:40
  • "As the code is now written, you need a mapping for struct Foo**" - I don't understand, there are typemaps `%typemap(in,numinputs=0) struct Foo** new_struct` and `%typemap(argout) struct Foo** new_struct`. – user2804197 Mar 28 '17 at 14:42
  • 1
    I should have said that typemaps for `struct Foo **` are the only ones that are applicable. I'll add that it seems doubtful that the mappings you're providing have the effect you want, as they appear to be copying a pointer value into and out of the Python object, whereas what you seem to want is to copy is the value of the (doubly indirected) actual struct. – John Bollinger Mar 28 '17 at 14:48
  • Indeed, If I dereference `$1` in the typemap it seems to work! – user2804197 Mar 28 '17 at 14:54
  • John Bollinger: Thanks for your help! Now that the typemap works, do you also have a suggestion where I could free the memory allocated for the struct or an alternative way to prevent the leak? – user2804197 Mar 28 '17 at 14:59
  • If you are in fact storing a C pointer in your object, then you must not free the data until you're done with it. You *could* provide (and wrap) another function that must be called to perform the deallocation (leaving your Python object in an invalid state), but that is grotesquely non-Pythonesque. It seems like your best bet might be to ditch SWIG, bite the bullet, and write a full-blown custom Python type. Then you can handle the deallocation when instances are destroyed. – John Bollinger Mar 28 '17 at 15:19
  • Alternatively, do not wrap (a pointer to) a dynamically allocated structure at all. *Convert* the struct to an ordinary Python object representation, so that Python's GC can handle the cleanup when appropriate. – John Bollinger Mar 28 '17 at 15:21
  • Actually, I think the `SWIG_NewPointerObj` does convert the struct to a Python object and the `%typemap(out) uint8_t a[4]` converts the array member to a Python bytes objects. I hope this way SWIG also takes care of deallocating the memory of the struct, but I'm not sure. – user2804197 Mar 28 '17 at 15:27
  • 1
    Not exactly. `SWIG_NewPointerObj` creates a proxy object around the pointer; it does not convert the pointed-to object. **But** that function does help you, now that you mention it. Its third argument indicates whether Python should take ownership of the pointer; if you pass a non-zero value then you should find that Python frees the pointer when the wrapper object is GCed. That would be a nice, clean option. – John Bollinger Mar 28 '17 at 15:53
  • Right, thanks! I edited the question accordingly, using SWIG_POINTER_OWN as the third argument. – user2804197 Mar 28 '17 at 16:15
  • I do not intend to write an answer for this question. However, it would be more appropriate for *you* to answer the question yourself than to edit your final code into the question, thus making it a non-question. – John Bollinger Mar 28 '17 at 16:21
  • I moved the solution to an answer. – user2804197 Mar 28 '17 at 16:40

1 Answers1

0

Here are example files (based on Accessing C struct array to Python with SWIG) which achieve what I want, i.e. create_struct() creates and initializes a struct, which can be used in Python. Thanks to John Bollinger for helping to fix bugs.

example.h

#include <stdint.h>

struct Foo
{
  uint8_t a[4];
};

void create_struct(struct Foo** new_struct);

example.c

#include <string.h>
#include "example.h"

void create_struct(struct Foo** new_struct){
    struct Foo* foo = (struct Foo*) malloc(sizeof(struct Foo));
    uint8_t tmp[4] = {0,1,2,3};
    memcpy(foo->a, tmp, 4);
    *new_struct = foo;
}

example.i

%module example
%{
#define SWIG_FILE_WITH_INIT
#include "example.h"
%}

// Define input and output typemaps for a

%typemap(in) uint8_t a[4] {
  if (!PyBytes_Check($input)) {
    PyErr_SetString(PyExc_TypeError, "Expecting a bytes parameter");
    SWIG_fail;
  }

  if (PyObject_Length($input) != 4) {
    PyErr_SetString(PyExc_ValueError, "Expecting a bytes parameter with 4 elements");
    SWIG_fail;
  }

  uint8_t res[4];
  char* bytes = PyBytes_AsString($input);
  int i;
  for (i=0; i<4; i++) {
    res[i] = (uint8_t) bytes[i];
  }

  $1 = res;
}

%typemap(out) uint8_t a[4] {
    $result = PyBytes_FromStringAndSize((char*) $1, 4);
}

/*
 * This fails with "Warning 453: Can't apply (struct Foo *OUTPUT). No typemaps are defined.":
 *   %apply struct Foo* OUTPUT {struct  Foo* new_struct };
 * So I'm trying to define typemaps for struct Foo*
 */

// This typemap suppresses requiring the parameter as an input.
%typemap(in,numinputs=0) struct Foo** new_struct (struct Foo* temp) {
  $1 = &temp;
}

%typemap(argout) struct Foo** new_struct {
    $result = SWIG_NewPointerObj(*$1, $descriptor(struct Foo*), SWIG_POINTER_OWN);
}

%include "example.h"

extern void create_struct(struct Foo** new_struct);

setup.py

#!/usr/bin/env python3

from distutils.core import setup, Extension

module1 = Extension('example', sources=['example.c', 'example.i'])
setup(name='Example', version='0.1', ext_modules=[module1])

test.py

#!/usr/bin/env python3

import example
foo = example.create_struct()
print("foo.a: %r" % foo.a)

Build and execute:

python3 setup.py build_ext --inplace && mv example.*.so _example.so && python3 test.py

The test code should print foo.a: b'\x00\x01\x02\x03'.

Community
  • 1
  • 1
user2804197
  • 354
  • 5
  • 13