4

I need to generate a sort comparator using a parameter I will only have at runtime in Cython.

Here is a minimal example:

from libcpp.algorithm cimport sort
import random

cpdef test_sort():
    cdef int *a = [1,2,3,4]
    cdef int z = random.randrange(5)
    sort(&a[0], &a[3], lambda x,y: abs(x-z) < abs(y-z)) # doesn't compile

I have tried lots of different approaches, but I can't get anything to compile. Is there any way to do this in Cython for native arrays and the libcpp sort function?

Update:

I tried @DavidW's suggestion to write the comparator in C++ but I am getting exactly the same error.

test_sort.pyx:

from libcpp cimport bool
from libcpp.algorithm cimport sort
from libcpp.functional cimport function

import random

cdef extern from "cmp.cpp":
    # I couldn't figure out how to declare the return type for this function
    # I tried function[bool(int, int)] make_cmp(int z) but Cython complained
    make_cmp(int z)

cpdef test_sort():
    cdef int *a = [1,2,3,4]
    cdef int z = random.randrange(5)
    sort(&a[0], &a[3], make_cmp(z))
    print(z)
    for i in range(4):
        print(a[i])

cmp.cpp:

#include <stdlib.h>

std::function<bool(int, int)> make_cmp(int z) {
    return [&z](int x, int y) { return abs(x-z) < abs(y-z); };

setup.py:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("*.pyx", language="c++")
)

run.py:

from test_sort import test_sort

test_sort()

$ python setup.py build_ext --inplace

Gives the following errors (among other output):

test_sort.cpp: In function ‘PyObject* __pyx_f_9test_sort_test_sort(int)’: test_sort.cpp:1069:33: error: cannot convert ‘std::function’ to ‘PyObject* {aka _object*}’ in assignment __pyx_t_2 = make_cmp(__pyx_v_z); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 13, __pyx_L1_error)

...

test_sort.cpp:1071:78: required from here /usr/include/c++/6/bits/predefined_ops.h:147:11: error: expression cannot be used as a function { return bool(_M_comp(*__it, __val)); }

error: expression cannot be used as a function is exactly what I get if I try to pass the python lambda into sort.

Imran
  • 12,950
  • 8
  • 64
  • 79
  • It's quite involved but this might be what you're after https://stackoverflow.com/questions/39044063/pass-a-closure-from-cython-to-c – DavidW Nov 23 '17 at 12:19
  • I saw that but it's mostly about passing a Python object to a C++ function, and maybe that can be avoided here? I was hoping there was some way to pass a C++ function to sort. – Imran Nov 23 '17 at 17:24
  • 1
    If you didn't need the closure on `z` then you could do it with a `cdef` function (which can just be passed as a function pointer). There's a couple of ways to pass callable Python objects (the C++ way linked, or `ctypes` callback) but neither is hugely simple. The easiest way by far would be to write the function in C++ (but if you need access to Python features in the function then that is harder) – DavidW Nov 23 '17 at 17:37
  • @DavidW I tried your last suggestion but couldn't get it to work. Please see my update – Imran Nov 23 '17 at 22:06
  • 2
    I can't test it right now but something like this should work: in C++ do `typedef std::function BIIFunc` (i.e. just give it a simple name), then in Cython do `cdef cppclass BIIFunc: pass` and `BIIFunc make_cmp(int z)`. Cython doesn't need to know that `BIIFunc` is actually a standard function so you keep all the nasty template stuff in C++ and out of Cython. – DavidW Nov 23 '17 at 22:12

0 Answers0