Pass python list to c++ extension using boost python

Question

I'm trying to write a c++ extension to replace the following python function in order to speed up my program

The python function looks like the following

def calc_dist(fea1, fea2):
    #fea1 and fea2 are two lists with same length
    ...

I wrote the function using c++ and boost python like follows:

#include <vector>
#include <boost/python.hpp>
double calc_dist(vector<double>& fea1, vector<double>& fea2)
{
    int len = fea1.size();
    double s=0;
    for(int i=0; i<len;i++){
        double p=fea1[i];
        double q=fea2[i];
        ...//calculating..
    }
    return s;
}
BOOST_PYTHON_MODULE(calc_dist)
{
    using namespace boost::python;
    def("calc_dist",calc_dist);
}

and compile the above cpp code into a .so file like

g++ calc_dist.cpp -shared -fPIC -o calc_dist.so -I /usr/include/python2.6 -lboost_python

and trying to use the .so in a python program, the import works fine, indicating the module can successfully imported.

However, whenever I pass two lists to the parameter to the function, python will give errors like

ArgumentError: Python argument types in
    calc_dist.calc_dist(list, list)
did not match C++ signature:
    calc_dist.calc_dist(std::vector<float, std::allocator<float> >,
                        std::vector<float, std::allocator<float> >)

can any one help me how to solve this problem? i.e pass a python list to c++ extension using boost?

Thanks a lot!

Use boost::python::list see [this question](http://stackoverflow.com/questions/3761391/boostpython-python-list-to-stdvector) — , May 18 '12 at 11:48

score 11 · Answer 1 · answered May 18 '12 at 11:58

11

Why did you write a function accepting std::vector if you want it to operate on a Python list? They're different things.

Boost.Python exposes python lists as the list class.

So, your function should look something like

double calc_dist(boost::python::list fea1, boost::python::list fea2)
{
    boost::python::ssize_t len = boost::python::len(fea1);
    double s=0;
    for(int i=0; i<len;i++){
        double p = boost::python::extract<double>(fea1[i]);
        double q = boost::python::extract<double>(fea2[i]);
        ...//calculating..
    }
    return s;
}

it's not tested, but hopefully is close enough to get you started ...

answered May 18 '12 at 11:58

Useless

64,155
6
88
132

I just put the example together from the online docs, so there should be enough information for you to make progress. – Useless May 21 '12 at 09:23
one would (perhaps naively) hope for an automatic conversion from a python list to a std::vector, no doubt. this code is more than enough to get started, since it includes a correct C++ argument type, and the necessary extract. thanks! – ofloveandhate Jan 15 '18 at 04:49
Why not automatic conversion to a `std::list`, or a `deque`, or something else? Python lists aren't vectors and don't behave like them. – Useless Jan 15 '18 at 09:58
Hi, thank you for your helpful answer, how about the performance of `boost::python::list`? – GoingMyWay Sep 02 '18 at 14:37
Probably depends what you're doing with it. As always, the best suggestion is to try it and see. – Useless Sep 03 '18 at 09:20
I am getting warning like the following: warning: ‘*((void*)& +24)’ may be used uninitialized in this function, when passing a boost::python list. How can I get rid of it? – Ryan Feb 09 '21 at 15:53
By posting it as its own question, with the minimal code necessary to reproduce the warning. That sort of looks like it might be a lambda. – Useless Feb 10 '21 at 10:28

Pass python list to c++ extension using boost python

1 Answers1