3

I have encountered the following strange behaviour of ctypes. When I convert a numpy array via ctypes to an int pointer some values are lost and additional zeros are added. More specifically, when I convert a numpy array with the following code 

from numpy import *
import ctypes
from numpy.ctypeslib import ndpointer

x = arange(1, 10)
xInt = x.ctypes.data_as(ctypes.POINTER(ctypes.c_int * len(x)))
xLong = x.ctypes.data_as(ctypes.POINTER(ctypes.c_long * len(x)))

Then I get the following confusing results:

print [i for i in xInt.contents]
[1, 0, 2, 0, 3, 0, 4, 0, 5]

print [i for i in xLong.contents]
[1, 2, 3, 4, 5, 6, 7, 8, 9]

What is happening?

python_noob
  • 31
  • 1
  • It looks to me like you're just getting a different view of the bytes. e.g. if numpy is creating the integers as longs. [On a 64 bit machine, sizeof long can be 8 whereas sizeof int is 4](http://stackoverflow.com/a/900245/748858). So, what seems to be happening is that you're cutting each number in half -- the first half contains the bits to construct the number (because you don't have any numbers bigger than maxint) and the second half is all zeros. – mgilson May 18 '15 at 16:56
  • Thanks! Shouldn't ctypes prevent that from happening? The first result isn't something you would expect, would you? – python_noob May 18 '15 at 17:09
  • 2
    I think it's what I'd expect. you're splitting the memory in half. So, if the number 1 is represented as `0x1000000` in a memory buffer -- Now when you say that the chunks of the buffer are only length 4 (c_int), when you construct your array of pointers, you get 2 pointers out of that 8 byte long buffer. On points to `0x1000` and the other to `0x0000`. – mgilson May 18 '15 at 17:12
  • I understand. But shouldn't ctypes know that python stores integers as long and correct for that? In other words: How can I know which type to use? What if I need an int array for my C function? – python_noob May 18 '15 at 17:16
  • @mgilson's comment is the answer itself. You can replicate the same behaviour by doing `x.view(np.int32)`, which will create an integer view of the array (4 bytes) which will make it double size and every even number will be zero (for numbers smaller than `2^31`). – Imanol Luengo May 18 '15 at 17:16
  • If you need an int array you can cast it to integer using `astype`. Or you can change the strides of the array to ignore every even 'number' as it will be the overflow from downcasting a long to int. – Imanol Luengo May 18 '15 at 17:18
  • @mgilson, it's 16 hex digits for a 64-bit integer: `(1).to_bytes(8, 'little') == b'\x01\x00\x00\x00\x00\x00\x00\x00'`. – Eryk Sun May 19 '15 at 00:26
  • @python_noob, ctypes should give you exactly what you asked for. It was given an address in memory and a type, which it accurately converted to a Python list. (BTW, it's simpler and more efficient to use a slice such as `xInt[0][:]` instead of the list comprehension.) – Eryk Sun May 19 '15 at 00:31

0 Answers0