PyArray_New or PyArray_SimpleNewFromData specify dimensions for a 3D array

Question

I have 1 dimensional float array (from C space) which I want to read inside python space with zero copy. So far what I have done (reading SO mostly) is:

// wrap c++ array as numpy array
//From Max http://stackoverflow.com/questions/10701514/how-to-return-numpy-array-from-boostpython
boost::python::object exposeNDarray(float * result, long size) {

    npy_intp shape[1] = { size }; // array size
    PyObject* obj = PyArray_SimpleNewFromData(1, shape, NPY_FLOAT, result);
    /*PyObject* obj = PyArray_New(&PyArray_Type, 1, shape, NPY_FLOAT, // data type
                              NULL, result, // data pointer
                              0, NPY_ARRAY_CARRAY_RO, // NPY_ARRAY_CARRAY_RO for readonly
                              NULL);*/
    handle<> array( obj );
    return object(array);
}

The PyArray_New commented part is equivalent in functionality to the PyArray_SimpleNewFromData one.

My problem is that this 1 dimensional array should actually be a 3 dimensional ndarray. I can control how my result float array is constructed and I want if possible for that continuous block of memory to be interpreted as 3 Dimensional array.

I think this can be done by specifying the shape variable but, I can't find any reference to how the memory is going to interpreted.

Say i need my array to look like: np.empty((x,y,z)). When i specify that in the shape variable, what section of my result array would make up the first dimension, what section the second and so on?

score 1 · Answer 1 · answered Jan 20 '17 at 14:21

1

You can use pybind11 for this. You can actually base yourself off an unit test that takes a c array and reads from it as a numpy view

answered Jan 20 '17 at 14:21

Felipe Lema

2,700
12
19

Unless I'm missing something, that's a simple 1D case? – XapaJIaMnu Jan 20 '17 at 14:48
The first argument is an initializer-list (or/for `std::vector`) of [shapes](https://github.com/pybind/pybind11/blob/5f07facef55cf93ff0d01106258255f6025d09ba/include/pybind11/numpy.h#L587). You can specify another shape for the same continuous memory array – Felipe Lema Jan 20 '17 at 15:45

score 1 · Accepted Answer · answered Jan 20 '17 at 18:24

There's documentation that describes the layout of a numpy array, e.g. https://docs.scipy.org/doc/numpy/reference/arrays.html

but maybe a simple example will help.

Let's make a 1d array of 24 integers, and reshape it to a 3d shape. If 'reshape' doesn't make sense, you'll need to review some array basics, including the notion of a view versus copy.

In [226]: arr = np.arange(24).reshape(2,3,4)
In [227]: arr
Out[227]: 
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])

A handy way of seeing the basic attributes of this array is this dictionary:

In [228]: arr.__array_interface__
Out[228]: 
{'data': (159342384, False),
 'descr': [('', '<i4')],
 'shape': (2, 3, 4),
 'strides': None,
 'typestr': '<i4',
 'version': 3}

data identifies the location of the data buffer that actually stores the values. In your construction this will be your C array (or a copy).

In this case it is a buffer of 96 bytes - 4 bytes per element. This buffer was created by the arange function, and 'reused' by the reshape.

In [229]: arr.tostring()
Out[229]: b'\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x05\x00\x00\x00\x06\x00\x00\x00\x07\x00\x00\x00\x08\x00\x00\x00\t\x00\x00\x00\n\x00\x00\x00\x0b\x00\x00\x00\x0c\x00\x00\x00\r\x00\x00\x00\x0e\x00\x00\x00\x0f\x00\x00\x00\x10\x00\x00\x00\x11\x00\x00\x00\x12\x00\x00\x00\x13\x00\x00\x00\x14\x00\x00\x00\x15\x00\x00\x00\x16\x00\x00\x00\x17\x00\x00\x00'

In [230]: len(_)
Out[230]: 96
In [231]: 24*4

the descr or arr.dtype identifies how the bytes are interpreted - here as a 4 byte integer, '

shape and strides determine how the 1d array is viewed - in this case as a 3d array.

In [232]: arr.strides
Out[232]: (48, 16, 4)
In [233]: arr.shape
Out[233]: (2, 3, 4)

This says that the first dimension (plane) is 48 bytes long, and there are 2 of them. The 2nd (each row) is 16 bytes long, and the step between column elements is 4 bytes.

By simply changing the strides and shape, a 1d array can be viewed as 2d, 3d. Even the array transpose is implemented by changing shape and strides (and another attribute, order) .

PyArray_New or PyArray_SimpleNewFromData specify dimensions for a 3D array

2 Answers2