0

Cython compilation fails silently whenever I put an import to a c++ library

%% cython -a --verbose
from libcpp.string cimport string

The compilation runs, but no annotations are shown and if you define a function the next cell doesn't find it.

The output:

[1/1] Cythonizing /Users/jorenvs/.ipython/cython/_cython_magic_88bf31fdc881aa1e28333abb92cb8079.pyx
building '_cython_magic_88bf31fdc881aa1e28333abb92cb8079' extension
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/lib/python3.7/site-packages/numpy/core/include -I/usr/local/lib/python3.7/site-packages/numpy/core/include -I/usr/local/lib/python3.7/site-packages/numpy/core/include -I/usr/local/lib/python3.7/site-packages/numpy/core/include -I/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c /Users/jorenvs/.ipython/cython/_cython_magic_88bf31fdc881aa1e28333abb92cb8079.c -o /Users/jorenvs/.ipython/cython/Users/jorenvs/.ipython/cython/_cython_magic_88bf31fdc881aa1e28333abb92cb8079.o
Joren Van Severen
  • 2,269
  • 2
  • 24
  • 30

1 Answers1

0

I found there are many reasons for the compilation failing and they're often silent. If you can't reproduce it using CLI cython, the easiest way to find out why it's failing, is running the verbose clang command manually. That should give you the exact error.

In my case it was this:

$ clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/lib/python3.7/site-packages/numpy/core/include -I/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c /Users/jorenvs/.ipython/cython/_cython_magic_a57fa1e560b400e6a28c993dfc83bfc5.c -o /Users/jorenvs/.ipython/cython/Users/jorenvs/.ipython/cython/_cython_magic_a57fa1e560b400e6a28c993dfc83bfc5.o

In file included from /Users/jorenvs/.ipython/cython/_cython_magic_a57fa1e560b400e6a28c993dfc83bfc5.c:630:
In file included from /usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
In file included from /usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1830:
/usr/local/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
 ^
/Users/jorenvs/.ipython/cython/_cython_magic_a57fa1e560b400e6a28c993dfc83bfc5.c:633:10: fatal error: 'ios' file not found
#include "ios"
         ^~~~~
1 warning and 1 error generated.

Which pointed me to this question: Compiling cython with gcc: No such file or directory from #include "ios"

Solution for me thus, add c++ language specification:

%% cython -a --verbose
#distutils: language = c++
from libcpp.string cimport string
Joren Van Severen
  • 2,269
  • 2
  • 24
  • 30
  • 1
    You should also look at the terminal you started Jupyter from - the error messages appear there. I think more recent versions of Cython (the 3.0alpha branch?) do capture the C/C++ output and send it to Jupyter (but I'm not completely sure) – DavidW Feb 18 '21 at 18:11
  • ha, didn't think of that. You're right, it does. I don't always have quick acces to it on remote servers though. – Joren Van Severen Feb 18 '21 at 21:43
  • 1
    you can use `%%cython -+` instead of adding `#distutils: language = c++`. – ead Feb 19 '21 at 18:46
  • Related: https://stackoverflow.com/q/57726729/5769463 Only that you don't see the error-message due to a bug. You could use wurlitzer on Linux to redirect the output to the notebook. – ead Feb 19 '21 at 18:50