13

I'm getting the following warning message when I try to use a ctypes array as a numpy array:

Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes, numpy
>>> TenByteBuffer = ctypes.c_ubyte * 10
>>> a = TenByteBuffer()
>>> b = numpy.ctypeslib.as_array(a)
C:\Python27\lib\site-packages\numpy\ctypeslib.py:402: RuntimeWarning: Item size
computed from the PEP 3118 buffer format string does not match the actual item s
ize.
  return array(obj, copy=False)
>>> b
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=uint8)

The code seems to be working, though. Is it a bad idea to ignore this warning?

Background: I'm calling a C DLL that produces data in real time. I need to pass the DLL a series of buffers to hold the data. While waiting for the next buffer to fill, I'd like to process the most recent buffer with numpy and save the result. I'm generating the buffers with code like above, and things seem to be working, but I don't want to sweep an important problem under the rug.

fish2000
  • 4,289
  • 2
  • 37
  • 76
Andrew
  • 2,842
  • 5
  • 31
  • 49

2 Answers2

14

It's a bug in Python. ctypes currently produces invalid PEP 3118 type codes, which Numpy notices: http://bugs.python.org/issue10746 http://bugs.python.org/issue10744

When such an inconsistency is present, Numpy skips using the PEP 3118 buffer interface, and falls back to the old (obsolete) buffer interface. This should work properly.

You can silence the warning using Python's warnings module. However, the warning may have a performance impact.

You can also try working around the issue by wrapping the ctypes object in buffer().

pv.
  • 33,875
  • 8
  • 55
  • 49
  • Excellent, thank you. Glad to know it's a known bug and not me doing something dumb. – Andrew Feb 11 '11 at 17:46
  • Can anyone comment on any change in performance by wrapping the ctypes objects with [`buffer()`](http://docs.python.org/2/library/functions.html#buffer)? Does this method reliably work? How does it differ from making a new numpy array from the ctype array EG: `np.array(my_ctype_array[:]).reshape([back, to, original])` other thank mangling the shape? I thought `ctypeslib.as_array()` was the recommended practice, but the warning message is really annoying and it will turn off non-tech users. Thx. – Mark Mikofski Aug 20 '13 at 22:14
  • 2
    Wrapping the ctypes objects with [`buffer()`](http://docs.python.org/2/library/functions.html#buffer) did **not** work for me EG: `new = numpy.ctypeslib.as_array(buffer(old))` returns ***TypeError: buffer() takes at least 1 argument (0 given)***. Can anyone add more clarity to the `buffer()` wrap option? Thanks! – Mark Mikofski Aug 20 '13 at 22:22
5

There is a more convenient way of doing this, which avoids the warning altogether:

Instead of creating the data as a ctypes array first and then converting it to a NumPy array, just create it as a NumPy array right away, and then use numpy.ctypeslib.ndpointer as type specifier in your ctypes prototype. As an example, let's say you have a C function called f which takes a char* and a size_t as arguments:

void f(char* buf, size_t len);

Your ctypes prototype would be

from numpy.ctypeslib import ndpointer
some_dll = ctypes.CDLL(...)
some_dll.f.argtypes = [ndpointer(numpy.uint8, flags="C_CONTIGUOUS"),
                       ctypes.c_size_t]
some_dll.f.restype = None

and you can call this function as

a = numpy.zeros(10, numpy.uint8)
f(a, a.size)
Sven Marnach
  • 574,206
  • 118
  • 941
  • 841