1

After having read a lot of documentation on numpy / cython I am still unable to create a numpy array from a pointer in cython. The situation is as follows. I have a cython (*.pyx) file containing a callback function:

cimport numpy

cdef void func_eval(double* values,
                    int values_len,
                    void* func_data):


    func = (<object> func_data)

    # values.: contiguous array of length=values_len

    array = ???

    # array should be a (modifiable) numpy array containing the
    # values as its data. No copying, no freeing the data by numpy.

    func.eval(array)

Most tutorials and guides consider the problem of turning an array to a pointer, but I am interested in the opposite.

I have seen one solution here based on pure python using the ctypes library (not what I am interested in). Cython itself talks about typed memoryviews a great deal. This is also not what I am looking for exactly, since I want all the numpy goodness to work on the array.

Edit: A (slightly) modified standalone MWE (save as test.cyx, compile via cython test.cyx):

cimport numpy

cdef extern from *:
    """
    /* This is C code which will be put
     * in the .c file output by Cython */
    typedef void (*callback)(double* values, int values_length);

    void execute(callback cb)
    {
      double values[] = {0., 1.};

      cb(values, 2);
    }

    """

    ctypedef void (*callback)(double* values, int values_length);

    void execute(callback cb);


def my_python_callback(array):
    print(array.shape)
    print(array)

cdef void my_callback(double* values, int values_length):
    # turn values / values_length into numpy array
    # and call my_pytohn_callback
    pass


cpdef my_execute():
    execute(my_callback)

2nd Edit:

Regarding the possible duplicate: While the questions are related, the first answer given is, as was pointed out rather fragile, relying on memory data flags, which are arguably an implementation detail. What is more, the question and answers are rather outdated and the cython API has been expanded since 2014. Fortunately however, I was able to solve the problem myself.

Firstly, you can cast a raw pointer to a typed MemoryView operating on the same underlying memory without taking ownership of it via

cdef double[:] values_view = <double[:values_length]> values

This is not quire enough however, as I stated I want a numpy array. But it is possible to convert a MemoryView to a numpy array provided that it has a numpy data type. Thus, the goal can be achieved in one line via

array = np.asarray(<np.float64_t[:values_length]> values)

It can be easily checked that the array operates on the correct memory segment without owning it.

hfhc2
  • 4,182
  • 2
  • 27
  • 56
  • Can you give a minimal runnable example? On the top of my head I don't even know how to set up some boilerplate to reproduce the scenario you're describing. – Nils Werner Oct 20 '19 at 18:32
  • Ok, that should do it. – hfhc2 Oct 20 '19 at 18:51
  • 2
    Possible duplicate of [Force NumPy ndarray to take ownership of its memory in Cython](https://stackoverflow.com/questions/23872946/force-numpy-ndarray-to-take-ownership-of-its-memory-in-cython) – ead Oct 20 '19 at 18:59
  • From the above duplicate you also should heed warning about right memory handling in my answer: https://stackoverflow.com/a/55959886/5769463 – ead Oct 20 '19 at 19:01
  • A small problem with your code: `values` can be deallocated while `array` is still alive. A better strategy is to have a Python-object which owns `values` and use `PyArray_SetBaseObject` to ensure the life-time of this object. – ead Oct 21 '19 at 20:52
  • 2
    Your second edit should be an self-answer. – ead Oct 21 '19 at 20:54

0 Answers0