Writing to new Python buffer interface

Question

I have implemented the new python buffer interface in C++ outlined here:

https://docs.python.org/2/c-api/buffer.html

I have implemented my Py_buffer struct and filled it in:

template<typename T>
static int getbuffer(PyObject *obj, Py_buffer *view, int flags) 
{
    flags;
    static const Py_ssize_t suboffsets[1] = { 0};
    view->buf           = (void*)(_Cast<T>(obj)->getbuffer());
    view->obj           = NULL;
    view->len           = _Cast<T>(obj)->getbuffersize();
    view->itemsize      = 1;
    view->readonly      = _Cast<T>(obj)->getreadonly();
    view->ndim          = 0;
    view->format        = NULL;
    view->shape         = NULL;
    view->strides       = NULL;
    view->suboffsets    = NULL;
    view->internal      = NULL;
    return 0;
}

I am creating my Python buffer class in Python and handing it to C++. I am getting a pyObject along with my Py_Buffer. So now my question is, how am I supposed to write and resize this pyBuffer in C++? I can get access to the pointer directly and a size. But if its a newly created buffer how do I tell it how much space I need? There does not seem to be any sort of resize function for me to call.

I can use: int result = PyBuffer_FromContiguous(&m_view, const_cast<void*>(data), pySize, 'A');

to add data to my buffer. But my buffer must already have the correct size or it wont write. I do not think this is the correct way to be using it anyway.

Cython is not an option.

I thought of a similar solution but am I allowed to reassign the view->buf pointer? Is this how things **are supposed to** be done? Is there any kind of documentation on how to implement a python write buffer? Other then just the method names. — marsh, Apr 10 '15 at 17:29
this won't reassign, this'll _extend_. I think, you can give this a try. — ForceBru, Apr 10 '15 at 17:31
""The function may move the memory block to a new location (whose address is returned by the function)"", The only way I can get the python data is through the function pointer that view->buf exposes. I cannot change the pointer that the function pointer is returning. — marsh, Apr 10 '15 at 17:33
I don't think you would resize the pybuffer directly, I think you would resize the associated pyobject, e.g. with `PyByteArray_Resize()` — 1.618, Apr 10 '15 at 18:57
You write "new python buffer interface" but that link goes to old Python 2, care to clarify which version is actually your target? — Ulrich Eckhardt, Apr 10 '15 at 19:43
I am using "The new-style Py_buffer struct" as indicated at the link not the "Old-style buffer objects" — marsh, Apr 10 '15 at 20:11
"The new buffer API has been backported to Python 2.6, and the memoryview object has been backported to Python 2.7. It is strongly advised to use them rather than the old APIs, unless you are blocked from doing so for compatibility reasons." — marsh, Apr 10 '15 at 20:11
Could you elaborate what you're doing exactly? What is your `Py_Buffer`, what is `pyBuffer`. Have you implemented a `PyObject` to be used as your buffer? — tynn, Apr 14 '15 at 08:04
Tynn the python c api knowledge is required here. These are all components of it. I feel like explaining how the python system works is not going to help either of us. — marsh, Apr 14 '15 at 12:58
Just asking, since both are not part of CPython. I wanted to know, if you have control over the implementation of the Buffer object, or if you only wanted to handle this task through the view. — tynn, Apr 14 '15 at 19:38
I am not sure what you mean the buffer object is not part of CPython? It is declared in PythonSDK\26\include\object.h line 177. typedef struct bufferinfo. I may be misunderstanding as I am fairly new to this. — marsh, Apr 14 '15 at 19:40

score 2 · Accepted Answer · answered Apr 14 '15 at 20:54

2

You shouldn't resize the Py_buffer directly, since it is just an interface to the data of a PyObject.

Instead, use PyByteArray_Resize() (or possibly _PyString_Resize()) on the underlying PyObject.

answered Apr 14 '15 at 20:54

1.618

1,765
16
26

Writing to new Python buffer interface

1 Answers1