1

Please read this post before answering: Pass a closure from Cython to C++

In the accepted answers, it is neatly shown how a python function is converted into a std::function using Boost Python.

Following this example I'm able to wrap functions taking an std::function as an argument and call them using a python function as input. However, this only works when the std::function parameters are primitives like int, double, string etc.

Any guidance on how to make this work for custom types as well will be highly appreciated.

Jon Petter
  • 87
  • 4
  • 1
    Can you give a complete but minimal example that demonstrates what you are trying to do? – metal Aug 29 '17 at 17:15

1 Answers1

1

This won't be a complete answer - it assumes you can fill in the gaps from my previous answer that the question was based on. Unfortunately it is a little bit more complicated than that case.


Just to define the problem - assume you have a parameter of a custom C++ class, like:

class cpp_class {
  // some non-trivial contents
};

and thus your C++ interface looks like this:

void call_some_std_func(std::function<void(cpp_class&)> callback) {
    callback(5,std::string("hello"));
}

The first thing to do is to write a Cython wrapper for your C++ class (in principle you could make a Boost Python wrapper instead). Here you need to make a choice of about "ownership" of the C++ object. The first choice is to make a copy:

cdef extern from "cpp_file.hpp":
  cppclass cpp_class:
    pass # details

cdef class CyWrapper:
  cdef cpp_class* ptr
  def __dealloc__(self):
    del self.ptr
  # other details following standard wrapper pattern

cdef public make_CyWrapper(cpp_class& x):
  obj = CyWrapper()
  obj.ptr = new cpp_class(x)
  return obj

I've created a wrapper class with a destructor that handles the memory and a publicly accessible constructor function that can be called from external code. This version is safe because your wrapper owns the object it holds and so there can be no writes to invalid memory. However, because it makes a copy, you can't make changes to the original C++ object.

A second option is to hold a pointer to an object you don't own. The code is basically identical except you remove the __dealloc__ and avoid making a copy in make_CyWrapper:

obj.ptr = &x // instead of new cpp_class(x)

This is unsafe - you need to ensure your C++ object outlives the Cython wrapper - but allows you to modify the object.

You could also imagine a few other options: you could take ownership of an existing object with your Cython wrapper (Such a scheme would have to pass by pointer rather than reference, or it could use move constructors); you could deconstruct your C++ class into a representation expressed in basic types and pass those to Python; you could use shared pointers to split the ownership; or you have a more elaborate way of marking your Cython wrapper as "invalid" once your held C++ instance is destructed.


What you do next depends on whether you're using Boost Python (for it's convenient, callable wrapping of Python objects) or if you're making your own version. (I showed both possibilities in the previous answer).

Assuming Boost Python, you need to do two things - tell it about the conversion and make sure that it imports the module that your wrapper is defined in (if you don't do this you get exciting segmentation faults)

struct convert_to_PyWrapper {
 static PyObject* convert(const cpp_class& rhs) {
    // the const_cast here is a bit dodgy, but was needed to make it work
    return make_CyWrapper(const_cast<cpp_class&>(rhs));
}
};

inline void setup_boost_python() {
    PyInit_your_module_name(); // named inityour_module_name in Python 2
    boost::python::to_python_converter<
        cpp_class,
        convert_to_PyWrapper>();
}

You need to make sure that your Python/Cython code calls "setup_boost_python" before attempting to use the callback (if you put it at module level it's done on import, which is ideal).

If you're following my "manual" scheme (avoiding the dependency on Boost Python) then you need to modify the call_obj Cython function that does the C++ to Cython type conversion.

cdef public void call_obj(obj, cpp_class& c):
    obj(make_CyWrapper(c))

You also need to ensure the wrapper Cython module is imported before use (otherwise you get segmentation faults). I did this in "py_object_wrapper.hpp" but providing it's done once somewhere you can place it where you like.

void operator()(cpp_class& a) {
    PyInit_your_module_name();
    if (held) {
        call_obj(held,a);
    }
}
DavidW
  • 29,336
  • 6
  • 55
  • 86
  • (This works with my illustrative implementation rather than the Boost Python one. I'll have a look at how to work it with Boost Python when I can) – DavidW Aug 29 '17 at 21:14
  • Thanks for the detailed answer David, but as indicated I was mostly after the Boost Python way of doing it. I guess I need something along this way: http://www.boost.org/doc/libs/1_62_0/libs/python/doc/html/faq/how_can_i_automatically_convert_.html – Jon Petter Aug 30 '17 at 06:32
  • I began to suspect you wanted the Boost Python way right after I'd written the answer for the other way... I think your link is the right idea: I think you make a converter in Cython (probably using a wrapper class as shown above), make a `cdef public` converter function to take in the C++ class and return the wrapper, and then use `boost::python::to_python_converter`. It'll probably be a day or too before I have time to look at it so if you or anyone else figures it out quicker please post it... – DavidW Aug 30 '17 at 06:48