11

Given a file docprep.pyx as simple as

from spacy.structs cimport TokenC

print("loading")

And trying to cythonize it via

cythonize -3 -i docprep.pyx

I get the following error message

docprep.c:613:10: fatal error: ios: No such file or directory
 #include "ios"
          ^~~~~
compilation terminated

As you can tell from the paths, this system has an Anaconda installation with Python 3.7. numpy, spacy and cython are all installed through conda.

AcK
  • 2,063
  • 2
  • 20
  • 27
tsorn
  • 3,365
  • 1
  • 29
  • 48
  • 6
    `ios` is a c++-header (I don't know where it comes from, but they should have called it `#include `), so you probably need to build it as c++, i.e `cython --cplus ...` – ead Nov 25 '18 at 20:39
  • Possible duplicate of [Using C++ STL maps in Cython](https://stackoverflow.com/questions/43241269/using-c-stl-maps-in-cython) – DavidW Nov 25 '18 at 20:47
  • @ead Thanks, that was the problem. Then I needed to fix 4 additional problems: 1) compile the `.cpp` file, not the `.c` file 2) use cython with `--embed` 3) Add `main` function to python program 4) use `-lpython3.7m` instead of `-lpython3.7`. Thanks! – tsorn Nov 25 '18 at 21:37
  • @DavidW This is not a duplicate. This questions is about standalone compilation. There's no `setup.py` or Python file calling into the `pyx`. – tsorn Nov 25 '18 at 23:04
  • Why is ios stuff included? I do not get it – The Unfun Cat Apr 04 '19 at 08:25
  • @TheUnfunCat It is something that cython includes – tsorn Apr 04 '19 at 09:08
  • 1
    In my case `--cplus` did not worked, I needed to add `language="c++"` in the setup.py file. – mountrix May 04 '19 at 07:16

2 Answers2

10

In my case, it worked using @mountrix tip, just add the language="c++" to your setup.py, an example:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
import numpy


extensions = [
    Extension("processing_module", sources=["processing_module.pyx"], include_dirs=[numpy.get_include()], extra_compile_args=["-O3"], language="c++")
]

setup(
    name="processing_module",
    ext_modules = cythonize(extensions),
)
Dielson Sales
  • 1,715
  • 1
  • 20
  • 25
  • 1
    You can also add the following line at the beginning of you `.pyx` file: `# distutils: language = c++` – Labo Apr 12 '20 at 18:13
6

<ios> is a c++-header. The error message shows that you try to compile a C++-code as C-code.

Per default, Cython will produce a file with extension *.c, which will be interpreted as C-code by the compiler later on.

Cython can also produce a file with the right file-extension for c++, i.e. *.cpp. And there are multiple ways to trigger this behavior:

  • adding # distutils: language = c++ at the beginning of the pyx-file.
  • adding language="c++" to the Extension definition in the setup.py-file.
  • call cython with option --cplus.
  • in IPython, calling %%cython magic with -+, i.e. %%cython -+.
  • for alternatives when building with pyximport, see this SO-question.

Actually, for cythonize there is no command line option to trigger c++-generation, thus the first options looks like the best way to go:

# distutils: language = c++

from spacy.structs cimport TokenC
print("loading")

The problem is that spacy/structs.pxd uses c++-constructs, for example vectors or anything else cimported from libcpp:

...
from libcpp.vector cimport vector
...

and thus also c++-libraries/headers are needed for the build.

ead
  • 32,758
  • 6
  • 90
  • 153
  • I think the `distutils: language` option is usually the way to go - the choice of language is really required the pyx file and therefore having to put the selector anywhere else is just making it harder to track what's happening. (It's definitely useful to list all the options though) – DavidW Mar 09 '21 at 14:17
  • distutils is deprecated https://docs.python.org/3/whatsnew/3.10.html#distutils-deprecated – user3673 May 04 '22 at 18:51