Cython: can't convert Python object to 'double *'

Question

I'm writting a Cython wrapper to a C function. I have a pxd file with the following signature:

double contr_hrr(int lena, double xa, double ya, double za, double *anorms)

When I try to call this from a pyx file

...
return contr_hrr(len(acoefs),a.origin[0],a.origin[1],a.origin[2],anorms2)

where anorms2 is a python list, I get the error message:

cython/ctwo.pyx:35:80: Cannot convert Python object to 'double *'

How do I pass a python list to a C function as a double array?

If you have the opportunity to test different methods, I would be very interested in knowing which is the fastest... — Gauthier Boaglio, Jun 09 '13 at 23:54
@Golgauth - I'll definitely give it a try. The code is 2x as slow as the old method where I directly wrapped the C routines using C. — Rick, Jun 10 '13 at 14:39
What was your final approach ? May be you could post your own answer on this... — Gauthier Boaglio, Jun 13 '13 at 08:16

score 10 · Accepted Answer · answered Jun 09 '13 at 23:02

10

cimport array:
```
from cpython cimport array
```
Create an array object from your list. array class constructor will do all the heavy lifting allocating memory and iterating over your list (could be any iterable actually).
```
cdef array.array anorms2_arr = array.array('d', anorms2)
```

Pass its data to your function:

return contr_hrr(.., anorms2_arr.data.as_doubles)

array is a standard Python module. Cython adds some special support on top, like buffer interface and direct access to the underlying memory block via arr.data.as_xxx. Unfortunately, this support is only documented here. You can also find some details about array usage in this recent thread.

answered Jun 09 '13 at 23:02

Nikita Nemkin

2,780
22
23

How recent is the array stuff? I'm getting "cython/ctwo.pyx:3:0: 'array.pxd' not found" when I try to cimport array. – Rick Jun 09 '13 at 23:32
@Rick I just checked the 0.19.1 (very last) stable release, and the file was there. Also tested the import with my current version `0.18` : it worked fine for me... Well, what's yours ? – Gauthier Boaglio Jun 09 '13 at 23:45
It was added in [Cython 0.17](http://wiki.cython.org/ReleaseNotes-0.17). The latest version is usually the best anyway. – Nikita Nemkin Jun 09 '13 at 23:48
Argh! I'm using cython 0.16 for some unknown reason. Think I had better upgrade. – Rick Jun 09 '13 at 23:55
Hmmm. Just tried to install cython through Macports, but the version is still showing at 0.16. – Rick Jun 10 '13 at 00:02
Okay. Just blew away the old v0.16 egg, and I'm now running v0.18. However, when I pass "anorms2.data.as_doubles" into the routine, I still get the Cannot convert Python object to 'double *'. – Rick Jun 10 '13 at 00:07
Documenting other approaches: I believe you can do this with numpy arrays as well, using the ctypes attribute as [documented here](x.ctypes.data_as(ctypes.POINTER(c_double))). Idea originally came from [this stackoverflow post](http://stackoverflow.com/questions/4495420/passing-numpy-arrays-to-c-code-wrapped-with-cython). I'll check out whether this is easier than the array approach (it does translate better to my workflow, since I use numpy normally, but not the array module). – Rick Jun 11 '13 at 18:54

Gauthier Boaglio · Answer 2 · 2013-06-09T23:29:26.517

I think that you cannot do otherwise, but convert it by yourself :

cimport cython
from libc.stdlib cimport malloc, free

...
cdef double *anorms
cdef unsigned int i;

anorms = <double *>malloc(len(anorms2)*cython.sizeof(double))
if anorms is NULL:
    raise MemoryError()

for i in xrange(len(anorms2)):
    anorms[i] = anorms2[i]

return contr_hrr(len(acoefs),a.origin[0],a.origin[1],a.origin[2],anorms)

If you had been in C++, this would have been different because

The following coercions are available:
Python type   =>   C++ type             => Python type
bytes              std::string             bytes
iterable           std::vector             list
iterable           std::list               list
iterable           std::set                set
iterable (len 2)   std::pair               tuple (len 2)

If you could switch to C++, you would have a direct translation from List[float] to vector<double> :

from libcpp.vector cimport vector

def py_contr_hrr(vector[double] anorms2, ...):
    ...
    return contr_hrr(len(acoefs),a.origin[0],a.origin[1],a.origin[2],anorms2)

And calling directly from Python side :

anorms2 = [12.0, 0.5, ...]
py_contr_hrr(anorms2, ....)

Source : http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html#standard-library

But I don't know if it is an option that you can consider... It depends on your project's constraints, of course.

EDIT : I didn't know about Nikita's way of doing (which is an elegant one, by the way), and I don't know ether which way is the best suitable concerning performances on big arrays.

Thanks very much for the post. I'm currently trying Nikita's way, and if that doesn't work I'll try this one. — Rick, Jun 10 '13 at 00:08

score 2 · Answer 3 · answered Jun 14 '13 at 10:01

What I finally did was to make sure that the anorms arrays were maintained as arrays in the python part of the code, and then followed Nikita's recipe to convert them on the fly to doubles using the .data.as_doubles property. If I do this, it appears to have very little overhead compared to doing everything natively in C.

Haven't yet experimented with the numpy approach, for a variety of mundane reasons.

Cython: can't convert Python object to 'double *'

3 Answers3

Linked