17

How does cython manage to cimport abs from libc.math

from libc.math cimport abs

if it is not there?

> grep abs Cython/Includes/libc/math.pxd
<nothing>

I even tried deleting everything from that file (I got a 0 length math.pxd) yet it somehow contrives to find abs there.

Update

This is my third question about cython:

In the first question I found out that libcpp.complex is not written quite correctly, @axil raised a ticket in cython github repo then made a pull request with my fix that replaces about 20% of complex.pyd lines. So it makes sense for cython user to look into includes directory from time to time.

The second question addressed the abs builtin which supposedly was overlooked by cython authors in that it doesn't translate python code to c at all and calls the original python abs function. The solution involves patching cython/compiler/Builtin.py file.

Now my third question is again about abs function, now imported from 'libc.math'. It does work, but they way it works looks really bizzare to me. In my opinion there's quite enough magic in Cython without importing a function from a module that doesn't have it.

Update 2:

It turns out that is that abs() does not get imported from lib.math. In fact, this import is just ignored:

 1: from libc.math cimport abs                   # nothing changes when I comment it out
 2: 
+3: def f():                                     # yellow
+4:     cdef double complex aaa = 1 + 2j         # white 
+5:     cdef double bbb = abs(aaa)               # yellow

__pyx_t_1 = __pyx_PyComplex_FromComplex(__pyx_v_aaa); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error)
__Pyx_GOTREF(__pyx_t_1);
__pyx_t_2 = PyNumber_Absolute(__pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error)
__Pyx_GOTREF(__pyx_t_2);
__Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
__pyx_t_3 = __pyx_PyFloat_AsDouble(__pyx_t_2); if (unlikely((__pyx_t_3 == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 5, __pyx_L1_error)
__Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_bbb = __pyx_t_3;

It should be noted though that @DavidW's patch from my previous question fixes it in the sense that last line becomes white (gets compiled to c). Yet I don't understand why cimporting abs is ignored.

Update 3

Yet another observation is that importing abs() from libcpp.complex works as expected:

# distutils: language = c++
from libcpp.complex cimport complex, abs

ctypedef complex[double] dcomplex

def f():
    cdef dcomplex aaa = dcomplex(1, 2)
    cdef double bbb = abs[double](aaa)
    return bbb

So it isn't a big deal for cython to cimport a builtin.

Community
  • 1
  • 1
Antony Hatchkins
  • 31,947
  • 10
  • 111
  • 111
  • I think it's due to the automatical optimization, http://docs.cython.org/en/latest/src/reference/language_basics.html#python-built-ins, built-in `abs` is translated , even in the `cimport`, to `PyNumber_Absolute`, `fabs`, `fabsf`, whatever. – oz1 Mar 29 '17 at 05:19
  • @oz1 Same thing happens when I write `from libc.math cimport abs as myabs` and `cdef double bbb = myabs(-10)`. Also when I inspect the c code for bbb initialization it is different if I use the mentioned import line or comment it out. It really uses `abs` from "math.h", not the builtin one. – Antony Hatchkins Mar 29 '17 at 06:14
  • Depending on the usage, `abs` will be translated to diffrent function call,`from libc.math cimport abs as myabs cdef double a = myabs(3+4j) cdef double b = myabs(-10) cdef double c = myabs(-10.0) print(a) print(b) print(c)`, just see the c code. BTW, I didn't get your real problem, does `abs` cause some error or bug? – oz1 Mar 29 '17 at 07:26
  • @oz1 Not sure what you're trying to say and how your comment applies to my question. After importing `abs` from `libc.math` it works correctly. And of course it is translated to a different function call depending on the argument type. Did I say the opposite somewhere? The question is how does cython import a function from a pxd module that doesn't have this function. – Antony Hatchkins Mar 29 '17 at 08:05
  • @oz1 Abs is the very first and the most simple function call I could think of, which is also needed in my work. My real problem is that cython, a tool for converting python code to c/c++ code doesn't convert `abs()` to c/c++. – Antony Hatchkins Mar 29 '17 at 08:11
  • @oz1 There's yet another cython issue: in your code `cdef double a = myabs(3+4j)` cython should've generated a compile error because there's no notion of complex numbers in `"math.h"`. Instead it inserts a python call with all the accompaining conversions. It is worth creating yet another so question or github bug report. Moreover, even builtin `abs()` is unable to cope with `double complex` - that is what my [second cython question](http://stackoverflow.com/questions/43067131/absdouble-complex-in-cython) is all about. – Antony Hatchkins Mar 29 '17 at 08:33
  • @AntonyHatchkins I don't know the answer to this but a bit of testing shows that `cimport` behaves oddly with names that are the same as Python builtins. Try compiling `from libc.math cimport ord` for example. I don't know what that means or why. – DavidW Mar 29 '17 at 08:49
  • @AntonyHatchkins Sorry to misunderstood your question. I don't know too much about the compiler-level of cython. I always try to use cython in a 'safe' way. eg: I never use `from libc.math cimport abs`:) But it does make sense to make this clear. Hope someone will give a good explanation – oz1 Mar 29 '17 at 09:01
  • 1
    @DavidW Yes, importing `ord` looks odd. It is missing from both `libc.math` and `"math.h"`, yet no error is generated and the builting odd keeps being used (like with `abs()`, the C version of it, so it's not really a big deal). I've also succeeded in importing double and even complex from literally empty `libc.math` without any warnings whatsoever :) Yet deleting that `math.pxd` file altogether breaks the compilation. As well as importing a non-builtin from empty file (like `sin()`). – Antony Hatchkins Mar 29 '17 at 09:46
  • @oz1 I hoped I won't have to go that deep either :) Thanks for you help anyway! – Antony Hatchkins Mar 29 '17 at 10:30
  • Related: Note that the current release version of Cython is missing a lot of declarations in `math.pxd` from `math.h`. This is fixed in Cython 3, as per [this commit](https://github.com/cython/cython/commit/4f9e2e37eb06de526938c428f35a355cdcc466f3#diff-408a441b237070a78e70a27b3bfe0462). – jmd_dk Sep 23 '20 at 09:00

2 Answers2

4

In C, the "built-in" or globally-available absolute-value function abs is only for integers (and labs for longs). The absolute-value function or floating-point numbers is fabs, found in "math.h".

This translates to using libc.stdlib.abs for integers, libc.math.fabs for doubles, and abs (built-in) for all Python objects. Only the built-in abs requires GIL.

Epic Wink
  • 796
  • 13
  • 18
1

Cython pxd files located in {Python_path}/.../site_packages/Cython/Includes So if look at libc/stdlib.pxd then you will see that stdlib.pxd got abs() and libcpp/complex.pxd have a template function abs for complex numbers template class. In libc/math.pxd you can find fabs() for floating point types.

Кирилл Солодских
  • 94
  • 6