0

I need to allocate memory for a large buffer, without actually initializing it (it will be used as a result buffer later and every location will be written to, so initialization is just a waste of time).

I can easily do this in Cython using:

cdef char *buff = <char *> malloc(n * sizeof(char))
if not buff :
        raise MemoryError()

The problem is I'm not sure how to return this as an ndarray to python (similar to numpy array). The documentation shows how to convert this to a list and return it, but not as an array.https://cython.readthedocs.io/en/latest/src/tutorial/memory_allocation.html

return [x for x in my_array[:number]]

Returning a list isn't very useful as I want this to be interpreted as a boolean array (this buffer will be used as a mask to filter out elements).

# Index and coefficient array are both numpy ND-arrays
index_array = index_array[mask]
coeff_array = coeff_array[mask]

how can I return the buffer as a boolean array after I calculated a mask?

OM222O
  • 310
  • 2
  • 11
  • There are ways to wrap it in an object and return that. But why but just associate a Numpy array instead of using `malloc`? – DavidW May 05 '22 at 06:16
  • I have found that numpy is actually pretty inefficinet! I'm not sure what's going on in the background but it's always slower than writing cython codes. Numpy.empty is supposed to return an array without initializing it, but I bet you it's slower than malloc (have no way of testing it for now since I don't know how to return the array) – OM222O May 05 '22 at 06:28
  • Just as a sanity check I checked the time taken to create a boolean array of size 4706878 (actual size of the mask I need) using np.empty and np.zeros and it took 16.2uS to create it using empty and 16.4uS to create it using zeros! something definitely seems wrong there. – OM222O May 05 '22 at 06:40
  • It's definitely slower than `malloc`. (I think it's slower than some other Python-based options like `array.array` too). I just suspect you'll most of the speed up you get when your create a Python object and wrap it into a Numpy array. I haven't actually tested this though, so could be wrong... – DavidW May 05 '22 at 06:43

0 Answers0