6

The python documentation on array clearly states that the array conforms to the buffer interface. It even suggest not using the buffer_info() method. But when I try to get a Py_Buffer from C/C++ code with PyObject_GetBuffer() or use python's memoryview, I get a failure.

For example, in python (I use version 2.7):

>>> a = array.array('c')
>>> memoryview(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot make memory view because object does not have the buffer interface

In fact, when I search python's code base, only bytearrayobject (bytearray), memoryobject (memoryview), and stringobject (str) have the required Py_TPFLAGS_HAVE_NEWBUFFER flag set on them. To my understanding, the documentation is wrong; array does not support the buffer interface.

I could use bytearray which supports the buffer interface, the problem is that I need the array's practical fromfile() method to read in a buffer that I can use in my C/C++ code.

Is there an alternative that would allow me to read a file into a buffer and use this buffer from C code, and not involve memory copies ? (I want to treat big binary files and copying is a less desirable option).

fish2000
  • 4,289
  • 2
  • 37
  • 76
David
  • 9,635
  • 5
  • 62
  • 68

2 Answers2

5

memoryview works only on objects that support the Python 3 buffer interface. array.array in Python 3 does, but it doesn't in Python 2.7. You might want to file a bug report for that. Simply use use bytearray (or str if you're using it read-only). Both support memoryview just fine.

Rosh Oxymoron
  • 20,355
  • 6
  • 41
  • 43
  • Agreed, as I said in my question. But... bytearray does not have the handy fromfile method. I guess I can use a string object to represent my binary data, but knowing a buffer (like bytearray) can be filled from file expands my options. There is also the unicode corner case of a string that would be eliminated if I could use another buffer type. – David Feb 02 '11 at 20:17
3

Python 2.6+ has two different buffer interfaces, just like it has two different class types: the classic version, and the Python 3 version.

From the Python/C API Reference Manual:

Two examples of objects that support the buffer interface are strings and arrays. The string object exposes the character contents in the buffer interface’s byte-oriented form. An array can only expose its contents via the old-style buffer interface. This limitation does not apply to Python 3, where memoryview objects can be constructed from arrays, too.

In Python 2.7 code, you can work with the old-style buffers using the buffer function, and new-style buffers using memoryview. Python 3 only supports the latter.

A similar distinction exists in the Python 2 C API; PyObject_GetBuffer is for the new buffer interface, PyBuffer_FromObject/PyBuffer_FromReadWriteObject is for the old buffer interface (and should work for arrays). See the above link for more information.

Søren Løvborg
  • 8,354
  • 2
  • 47
  • 40