Pybind Numpy access 2D / ND arrays

Question

New to pybind - read the documentation but I do not grasp how to apply it to 2D arrays.

I have two arrays storing 3d coordinates shape = (10,3)

a = np.zeros(shape=(10,3))
b = np.ones(shape=(10,3)) * 3
c = a + b

Now, using pybind, how do I perform this operation in C++ working on the numpy arrays?

In some documentations I read to access the elements with the [] operator, in others with (). How do assign the 3D vector? How would I get the pointer to the array element to use strides for assignment - or does it have an operator?

I don't get what you are asking. It probably depends on how to treat these using C++. E.g. as std-vector or for example as Eigen-based matrices/arrays. These both accept different styles of indexing. Your example also consists of 2d-arrays only. What 3d-vector (which also sounds bad in a mathematical way) are you talking about? — sascha, Mar 31 '18 at 04:23

Christian · Accepted Answer · 2021-02-23T07:35:12.373

19

PyBind is awesome, shout out to the authors/maintainers! You have an almost working example here.

Adapted to your problem it would give something like (edited answer after El Dude's comment):

#include <iostream>
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>


namespace py = pybind11;


py::array_t<double> add_arrays(py::array_t<double> input1, py::array_t<double> input2) {
  py::buffer_info buf1 = input1.request();
  py::buffer_info buf2 = input2.request();

  if (buf1.size != buf2.size) {
    throw std::runtime_error("Input shapes must match");
  }

  /*  allocate the buffer */
  py::array_t<double> result = py::array_t<double>(buf1.size);

  py::buffer_info buf3 = result.request();

  double *ptr1 = (double *) buf1.ptr,
         *ptr2 = (double *) buf2.ptr,
         *ptr3 = (double *) buf3.ptr;
  int X = buf1.shape[0];
  int Y = buf1.shape[1];

  for (size_t idx = 0; idx < X; idx++) {
    for (size_t idy = 0; idy < Y; idy++) {
      ptr3[idx*Y + idy] = ptr1[idx*Y+ idy] + ptr2[idx*Y+ idy];
    }
  }
 
  // reshape array to match input shape
  result.resize({X,Y});

  return result;
}


PYBIND11_MODULE(example, m) {
        m.doc() = "Add two vectors using pybind11"; // optional module docstring

        m.def("add_arrays", &add_arrays, "Add two NumPy arrays");
}

That I built on linux with python2.7 and gcc v5.4 using (I had to use a slightly different command than provided in the doc, because Python.h wasn't found, hence I added the link to python 2.7)

c++ -O3 -Wall -shared -std=c++11 -fPIC -I/usr/include/python2.7 -lpython2.7 `python -m pybind11 --includes` example.cpp -o example`python-config --extension-suffix

And you'd call it from python with

import numpy as np
import example # [bad] name I chose for my compiled module

a = np.zeros((10,3))
b = np.ones((10,3)) * 3 
c = example.add_arrays(a, b)

print c

Hope it helps.

EDIT - I've created a github repository containing a few complete examples based on PyBind11 that should compile on all platforms.

edited Feb 23 '21 at 07:35

answered Apr 06 '18 at 13:16

Christian

1,162
12
21

Reshape should be possible by adjusting size & strides in the buffer object? – El Dude Apr 06 '18 at 17:55
You're right, I missed it but there's a `resize` function on the `array_t` itself, I was trying to look into `buffer_info`. I'll edit my answer for the change, thank you. – Christian Apr 09 '18 at 07:22
Well, I found the example code before the post and the solution before your reply... But here you are. – El Dude Apr 10 '18 at 23:28
I get a compilation error on this line: py::buffer_info result = py::array_t(buf1.size); – Yonatan Simson Jul 09 '18 at 11:26
Sorry it is my mistake, it should be `py::array_t result = py::array_t(buf1.size);` I'll edit the answer. You can find this code on [my github page](https://github.com/cjaques/pybind_examples) – Christian Jul 09 '18 at 11:34
I get a strange problem with this approach! Basically the index positions of values in my arrays `a` and `b` get shuffled around, so `c` has the right values, but not in the right places. Any clue how to order the value positions in memory so this doesn't happen? – Ulf Aslak Feb 08 '20 at 16:38
Strange indeed. Do you have this shuffle using exactly this example? If not, can you print the type of data you send from python to C++? – Christian Feb 09 '20 at 13:00
the link is dead – Stepan Yakovenko Oct 19 '20 at 23:56
I just fixed it, hope it helps – Christian Oct 22 '20 at 09:16

El Dude · Answer 2 · 2018-04-06T22:15:24.700

The trick is to use the buffer class. It's well hidden / convoluted into the documentation and examples, but it's mentioned (@Christian 's post).

Buffers contain a pointer to the data as well as strides and others array parameters. Essentially the numpy header accessed through request method. Easy to use from there on, but finding it is bit of a pain since the example uses beautiful C11 auto type to explain this usage.

Pybind Numpy access 2D / ND arrays

2 Answers2

Linked