How to write auto-populate the parameter list of a class member method into a Variadic arguments?

Question

I recently wrote an extension to Python 3 in C++, but I encountered some trouble when I called C++ in python.

I don't know how to encapsulate the code below, without the need to write callback function repeated every time?

I am considering binding a callback function and parameter list in some form, but I don't know how to do this.

Here is my main code:

class TestClass
{
    PyObject_HEAD
public:

    int add(int a, int b) {
        return (a + b);
    }

};

// ... Ignore some details here ...

static PyObject* function1(TestClass *self, PyObject* args) {

    // How to changed the following code to take the value from PyArg_ParseTuple
    // through the binding function and parameter list?
    int a, b;
    if (!PyArg_ParseTuple(args, "ii", &a, &b))
    {
        return nullptr;
    }

    // How to changed the following code to the return type of the binding function?
    return Py_BuildValue("i", self->add(a, b));
}

Is it possible to implement a call like BINDING_FUNCTION(TestClass::add); in some way?

Answer 1

Yes, this is possible, but it takes a nontrivial amount of code. Too much, I think, to easily fit in an answer here.

The good news is that others have already written the code you need and put them in libraries. The ones I know are pybind11 (modern) and boost.python (somewhat older)

You could either look how these libraries do this, or actually use these libraries.

Approximate steps required if you really want to do this yourself:

• Create generic functions to create format strings from function argument types and return type
• Create template meta function to define tuple type to hold the arguments of a given (member-) function
• Implement generic function that can fill such a tuple using PyArg_ParseTuple(). This would use the format-string-generating function and the tuple definition
• implement a generic function that takes an object, a (member-) function pointer and a tuple with arguments and that will call the member function with the given arguments. SO has answers on how to do this.

Typically, you'd want to specialize that last function for void return type functions and you might want to write some extra code if you want to support free functions in addition to member functions.

The code below is for educational purposes only because it's a very much simplified implementation of the steps described above. The libraries mentioned before implement the many corner cases that this code does not address.

#include <iostream>
#include <sstream>
#include <type_traits>
#include <tuple>

// some object we want to wrap
struct Duck
{
    int MemberFunc( int x, int y, float w)
    {
        std::cout << "Member function called\n";
        return x+ w * y;
    }
};


// PART 1: create format strings for function argument- and return types

// "excercise for the reader": implement these overloads for all supported types
template<typename T> struct Tag{};
const char *GetTypeFormat( const Tag<int>&)
{
    return "i";
}
const char *GetTypeFormat( const Tag<float>&)
{
    return "f";
}

// create a format string from a list of argument types
template< typename... Args>
void GetTypeFormats( std::ostream &strm)
{
    (void)(int[]){0, ((strm << GetTypeFormat(Tag<Args>{})),0)...};
}

// this is quite inefficient because it creates the format string at 
// run-time. Doing this as constexpr is an interesting challenge
// ("...for the reader")
template< typename R, typename Class, typename... Args>
std::string GetArgumentFormats( R (Class::*f)(Args...))
{
     std::stringstream strm;
     GetTypeFormats<Args...>( strm);
     return strm.str();
}

template< typename R, typename Class, typename... Args>
std::string GetReturnFormat( R (Class::*f)(Args...))
{
     std::stringstream strm;
     GetTypeFormats<R>( strm);
     return strm.str();
}

// PART 2: declare std::tuple-type to hold function arguments

// given a list of types that could be function parameter types, define a 
// tuple type that can hold all argument values to such a function
// THIS IS VERY MUCH A SIMPLIFIED IMPLEMENTATION
// This doesn't take pointer types into account for instance.
template< typename F>
struct ArgumentTuple {};

template< typename R, typename Class, typename... Args>
struct ArgumentTuple<R (Class::*)( Args...)>
{
    using type = std::tuple<
        typename std::remove_cv<
            typename std::remove_reference<Args>::type>::type...>;
};


// for demo purposes. emulate python binding functions
using PyObject = void;
bool PyArg_ParseTuple( PyObject *, const char *, ...) {}
template< typename T>
PyObject *Py_BuildValue( const char*, T ){}

// PART 3: given some function pointer, obtain arguments from a PyObject
template<typename F, size_t... Indexes>
auto FillTuple( PyObject *obj, F f, std::index_sequence<Indexes...>) -> typename ArgumentTuple<F>::type
{
    using std::get;
    typename ArgumentTuple<F>::type arguments;
    // no error checking whatsoever: "exercise for the reader" 

    PyArg_ParseTuple( obj, GetArgumentFormats( f).c_str(), &get<Indexes>( arguments)...);
    return arguments;
}

template< typename R, typename Class, typename... Args>
auto FillTuple( PyObject *obj, R (Class::*f)(Args...))
{
    return FillTuple( obj, f, std::index_sequence_for<Args...>{});
}

// PART 4, call a member function given a tuple of arguments

// helper function
template< 
    typename R, 
    typename Class, 
    typename MF, 
    typename ArgumentTuple, 
    size_t... Indexes>
R Apply( Class &obj, MF f, ArgumentTuple &args, std::index_sequence<Indexes...>)
{
    using std::get;
    return (obj.*f)( get<Indexes>( args)...);
}

// Apply a (member-) function to a tuple of arguments.
template<
    typename R,
    typename Class,
    typename ArgumentTuple,
    typename... Args>
R Apply( Class &obj, R (Class::*f)( Args...), ArgumentTuple &args)
{
    return Apply<R>( obj, f, args, std::index_sequence_for<Args...>{});
}


// LAST PART: glue everything together in a single function. 
#define BIND_MEMBER_FUNCTION( class_, memberfunc_)                  \
    PyObject *Call##class_##memberfunc_( class_ *self, PyObject *args)\
    {                                                               \
        /* no error checking whatsoever: "exercise for the reader"*/\
        auto arguments = FillTuple( args, &class_::memberfunc_);    \
        /* deal with void-returning functions: yet another EFTR */  \
        return Py_BuildValue(                                       \
            GetReturnFormat( &class_::memberfunc_).c_str(),         \
            Apply( *self, &class_::memberfunc_, arguments));        \
    }                                                               \
    /**/

BIND_MEMBER_FUNCTION( Duck, MemberFunc);