Wrap a function that takes a struct of optional arguments using kwargs

Question

In C it's not uncommon to see a function that takes a lot of inputs, many/most of which are optional group these up in a struct to make the interface cleaner for developers. (Even though you should be able to rely on a compiler accepting at least 127 arguments to a function nobody actually wants to write that many, especially as C has no overloading or default function argument support). As a hypothetical example we could consider the following struct/function pair (test.h) to illustrate the problem:

#include <stdbool.h>

typedef struct {
  const char *name;
  void *stuff;
  int max_size;
  char flags;
  _Bool swizzle;
  double frobination;
  //...
} ComplexArgs;

void ComplexFun(const ComplexArgs *arg) {}

When it comes to wrapping this using SWIG we can get something working quickly using:

%module test

%{
#include "test.h"
%}

typedef bool _Bool;

%include "test.h"

That works and we can use it as follows:

import test

args=test.ComplexArgs()
args.flags=100;
args.swizzle=True
test.ComplexFun(args)

But that isn't exactly Pythonic. A Python developer would be more accustomed to seeing kwargs used to support this kind of calling:

import test

# Not legal in the interface currently:
test.ComplexFun(flags=100, swizzle=True)

How can we make that work? The SWIG -keyword command line option doesn't help either because there's only one actual argument to the function.

Answer 1

Normally in Python the way to modify function arguments and return values is to use a decorator. As a starting point I sketched out the following decorator, which solves the problem:

def StructArgs(ty):
  def wrap(f):
    def _wrapper(*args, **kwargs):
      arg=(ty(),) if len(kwargs) else tuple()
      for it in kwargs.iteritems():
        setattr(arg[0], *it)
      return f(*(args+arg))
    return _wrapper
  return wrap

It has some further neat properties when written like that:

1. It also doesn't break the syntax for calling the function directly with the single struct argument
2. It can support functions with mandatory positional arguments and a struct full of optional arguments as the last argument. (Although it can't use kwargs syntax for mandatory non-struct arguments currently)

The question then becomes one of simply applying that decorator to the right function inside the SWIG generated Python code. My plan was to wrap it up in the simplest possible macro I could, because the pattern is repeated across the library I'm wrapping lots. That turned out to be harder than I expected though. (And I'm apparently not the only one) I initially tried:

1. %feature("shadow") - I was pretty sure that would work, and indeed it does work for C++ member functions, but it doesn't work for free functions at global scope for some reason I didn't figure out.
2. %feature("autodoc") and %feature("docstring") - optimistically I'd hoped to be able to abuse them slightly, but no joy
3. %pythoncode right before the SWIG sees the function declaration on the C side. Generates the right code, but unfortunately SWIG immediately hides the function we decorated by adding ComplexFun = _test.ComplexFun. Couldn't find a way around it for quite a while.
4. Use %rename to hide the real function we call and then write a wrapper around the real function which was also decorated. That worked, but felt really inelegant, because it basically made writing the above decorator pointless instead of just writing it in the new wrapper.

Finally I found a neater trick to decorate the free function. By using %pythonprepend on the function I could insert something (anything, a comment, pass, empty string etc.) which as enough to suppress the extra code that was preventing #3 from working.

The final problem I encountered was that to make it all work as a single macro and get the position of the %pythoncode directive right (also still permit %includeing of the header file which contained the declaration) I had to call the macro before %include. That necessitated adding an additional %ignore to ignore the function if/when it's seen a second time in an actual header file. However the other problem it introduced is that we now wrap the function before the struct, so inside the Python module the type of the struct we need the decorator to populate isn't yet known when we call the decorator. That's easily enough fixed by passing a string to the decorator instead of a type and looking it up later, in the module globals().

So with that said the complete, working interface that wraps this becomes:

%module test

%pythoncode %{
def StructArgs(type_name):
  def wrap(f):
    def _wrapper(*args, **kwargs):
      ty=globals()[type_name]
      arg=(ty(),) if kwargs else tuple()
      for it in kwargs.iteritems():
        setattr(arg[0], *it)
      return f(*(args+arg))
    return _wrapper
  return wrap
%}

%define %StructArgs(func, ret, type)
%pythoncode %{ @StructArgs(#type) %} // *very* position sensitive
%pythonprepend func %{ %} // Hack to workaround problem with #3
ret func(const type*);
%ignore func;
%enddef

%{
#include "test.h"
%}

typedef bool _Bool;

%StructArgs(ComplexFun, void, ComplexArgs)

%include "test.h"

This then was enough to work with the following Python code:

import test

args=test.ComplexArgs()
args.flags=100;
args.swizzle=True
test.ComplexFun(args)

test.ComplexFun(flags=100, swizzle=True)

Things you'd probably want to do before using this for real:

1. With this decorator and kwargs as currently written it's pretty hard to get any kind of TypeError back. Probably your C function has a way of indicating invalid combinations of inputs. Convert those into TypeError exceptions for Python users.
2. Adapt the macro to support mandatory positional arguments if needed.

Answer 2

Flexo's decoration is very impressive. I came across this problem for myself and hesitate to propose my solution except that it has one saving grace: simplicity. Also, my solution was for C++, but you might be able to modify it for C.

I declare my OptArgs struct like this:

struct OptArgs {
  int oa_a {2},
  double oa_b {22.0/7.0};
  OptArgs& a(int n)    { a = n; return *this; }
  OptArgs& b(double n) { b = n; return *this; }
}

with the intention of calling the constructor from C++ with MyClass(required_arg, OptArgs().b(2.71)) for example.

Now I use the following in the .i file to move the SWIG-generated constructor out of the way and unpack keyword arguments:

%include "myclass.h"
%extend MyClass {
    %pythoncode %{
        SWIG__init__ = __init__
        def __init__(self, *args, **kwargs):
            if len(kwargs) != 0:
                optargs = OptArgs()
                for arg in kwargs:
                    set_method = getattr(optargs, arg, None)
                    # Deliberately let an error happen here if the argument is bogus
                    set_method(kwargs[arg])
                args += (optargs,)
            MyClass.SWIG__init__(self, *args)
    %}
};

It's not perfect: it relies on the extension happening after the __init__ generated by SWIG has been declared, and is python-specific, but seems to work OK and is very, very simple.

I hope that's helpful.