Using SWIG for functions with functions as input parameter

Question

Consider the following minimal example: I have a C++ function 'myfun' that takes a number and another function as input, and returns the function evaluated at x:

double myfun(double x, double (*f)(double y))
{ return f(x); }

I store this code in 'test.cpp'. The code in the corresponding header file 'test.hpp' is simply

double myfun(double x, double (*f)(double y));

Now, I try to access 'myfun' from Python using SWIG. The 'swig.i' file looks as follows:

%module test
%{
#include "test.hpp"
%}
double myfun(double x, double (*f)(double y));

Whenever I try to call this function in Python, e.g. with

from test import myfun
myfun(2, lambda y: y**2)

I receive the error:

in method 'myfun', argument 2 of type 'double (*)(double)'

So my question is simply: How to modifiy the 'swig.i' file such that I can pass any (suitable Python) function to 'myfun'? (The question is somewhat related to this post How to wrap a c++ function which takes in a function pointer in python using SWIG' but the difference is that here I do not specify the input function in C++, but instead I want to keep it as a free 'parameter').

Answer 1

Callbacks require extra treatment when writing bindings and this is not (yet) supported in SWIG and apparently there are no real workarounds.

The main point is that SWIG is about calling C from Python (and other languages), but not calling Python from C.

We are using SIP instead that is a much more sophisticated binding generator (that is however C++/Python specific). SIP supports callbacks and even inheriting Python classes from C++ classes.

The tool is pretty powerful and is the engine behind PyQt binding.

Note however that a callable in Python is more than a function in C++ because it can contain a context (it can be a closure). In C++ code like:

int callF(int (*f)(int, int), int a, int b) {
    return f(a, b);
}

can only call a pure context-free function so passing it a general Python callback is impossible because there's not enough information (the only solution would be allocating pre-compiled trampolines or generating C++ code at runtime).

If you can control both sides however a workaround is to create a C class containing the callback and derive it in Python. For example:

struct CBack {
    virtual int operator()(int x, int y) const {
        return x+y;
    }
    virtual ~CBack() {}
};

int callF(const CBack& f,
          int xv, int yv)
{
    return f(xv, yv);
}

and then deriving a Python class from CBack and passing that:

class Add(CBack):
    def __call__(self, x, y):
        return x + y

class Mul(CBack):
    def __call__(self, x, y):
        return x * y

print callF(Add(), 3, 7) # will print 10
print callF(Mul(), 3, 7) # will print 21

You can in this case also pass a (wrapped) lambda

def cback(f):
    class PyCBack(CBack):
        def __call__(self, x, y):
            return f(x, y)
    return PyCBack()

print callF(cback(lambda x, y: x*y), 3, 7)

In other words one impedance problem is that a function for C++ is not the same as a function for Python (a function for Python can also have context).

A full example can be downloaded from here.

Answer 2

According to the documentation (emphasis mine):

Although SWIG does not normally allow callback functions to be written in the target language, this can be accomplished with the use of typemaps and other advanced SWIG features.

This is the the suggested chapter if you want to go deeper into the topic.
Anyway, it looks like it is not an intended key feature of the tool.
The best you can do probably is to get a solution that is designed around Python and won't work with other target languages, but it is not usually what an user of Swig wants from it.

Note that you can still write your callbacks in C/C++ and export them as a constants, then use them from the target languages.
See the above me documentation for further details.