Using Python objects in C++

Question

I'm writing a code that calculates the images of nonlinear maps using methods from interval analysis, applies a minkowski sum and repeats for an arbitrary number of iterations.

I've written a working code in Python, however I would like to be able to implement some of the more iteration/recursion intensive parts of the algorithm in C++ to benefit from the increased speed. I have used Cython in the past with great results, but I'd like to practice my C++.

Also, my objects are complicated enough that I'd rather avoid having to implement them in C++ (baby steps!).

So my questions are:

1) Does using Python objects in C++ prevent any improvement in efficiency?

2) If not, is it possible to use Cython to wrap a C++ function which iterates/recurses over a python object?

To be more specific, I have a recursive algorithm that recurses over the left and right children of a BST (though it's a fairly heavily modified BST so I'd rather not get bogged down with the details of implementing it in C++), however the runtime is quite prohibitive, so I'd like to write it in C++.

Answer 1

Yes. You will get a speed-up over pure python but not on par with the increase you'd get if you were using pure C/C++. If you want to handle Python Objects you'll need to do it via the Python C/API; this adds overhead to the execution, the price you must pay for being allowed to interact with Python.

Do note this involves a lot of complexity since you need to be familiar with the API and read up what functions do with object references, how to create Lists, pack Tuples et cetera. You can skip through all these if you just create a couple of public Cython cdef functions that wrap the methods on your objects. This generates all the CPython code that handles these for you.

A little example in which a silly object is wrapped and embedded might look like this (note, I'm using .c for this, c++ has similar steps):

class PyClass(object):

    def __init__(self):
        self.data = []

    def add(self, val):
        self.data.append(val)

    def __str__(self):
        return "Data: " + str(self.data)

cdef public object createPyClass():
    return PyClass()

cdef public void addData(object p, int val):
    p.add(val)

cdef public char* printCls(object p):
    return bytes(str(p), encoding = 'utf-8')

Compiling with cython pycls.pyx (use --cplus for c++) will generate a .c and .h file containing the source and the function declarations respectively. All you need to do now is create a main.c file that starts up Python and you're ready to call these functions:

#include "Python.h"   // Python.h always gets included first.
#include "pycls.h"    // Include your header file.

int main(int argc, char *argv[])
{
    Py_Initialize();   // initialize Python
    PyInit_pycls();    // initialize module (initpycls(); in Py2)
    PyObject *obj = createPyClass();
    for(int i=0; i<10; i++){
        addData(obj, i);
    }
    printf("%s\n", printCls(obj));
    Py_Finalize();
    return 0;
}

Compiling this with the proper flags (which you can obtain from python3.5-config of python-config [Py2]):

gcc pycls.c main.c -L$(python3.5-config --cflags) -I$(python3.5-config --ldflags) -std=c99

Will create your executable which interacts with your object:

./a.out
Data: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

All this is done by using Cython along with the public keyword that generates the .h header file. You could alternatively just compile a python module with Cython and create the header/handle the additional boilerplate yourself. Since I don't think you want to get boggled down with C-API learning, this shouldn't be the way to go.

As @freakish states in his comment, it would be ideal to extract the data (numpy has a C-API you can use for this) and work on it in pure C++. Generally if you work your loops in C/C++ and perform the grunt work there, you'll get good speed ups.