4

I'm trying to compile both C and C++ sources at the same time in Cython. This is my current setup:

-setup.py

from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
import os

language = "c++"
extra_compile_flags = ["-std=c++17"]
os.environ["CC"] = "clang++"

ext_modules = [
    Extension(
        name="Dummy",
        sources=["mydummy.pyx", "source1.cpp","source2.c"],
        language=language,
        extra_compile_args=extra_compile_flags,
   )
]

ext_modules = cythonize(ext_modules)

setup(
    name="myapp",
    ext_modules=ext_modules,
)

-compile command:

python3 setup.py build_ext --inplace --verbose

In the log I get the following message:

clang++ -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I. -I/usr/include/python3.6m -I/usr/include/python3.6m -c /path/source2.c -o build/temp.linux-x86_64-3.6/./path/source2.o -std=c++17 -O3
clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]

The compilation goes trough, but the warning looks pretty nasty. How can I get rid of it? The naive solution

os.environ["CC"] = "clang"
os.environ["CXX"] = "clang++"

is failing with error: invalid argument '-std=c++17' not allowed with 'C'

Rexcirus
  • 2,459
  • 3
  • 22
  • 42

1 Answers1

5

gcc (or clang) is just a frontend and it calls the appropriate compiler (c- or c++-compiler) given the file-extension of the compiled file (.c means it should be compiled with c-compiler and .cpp means it should be compiled with c++-compiler), see for example this similar SO-issue.

Thus there is no need to set the compiler to "clang++", as it will happen automatically.

However, cython needs to know that it has to produce a cpp-file (along with accepting c++-syntax) and not a c-file. Also the linker has to know, that it has to link against cpp-libaries (it is done automatically if g++/clang++ is used for linking). Thus we need to add language = "c++" to the Extension's definition, as suggested in DavidW's answer - this will take care of the last two problems.

The remaining problem is, that we would to like use different compiler options (-std=c++17 only for cpp-files) for different source-files. This can be achieved using a less known command build_clib:

#setup.py:

from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension

ext_modules = [
    Extension(
        name="mydummy",
        sources=["mydummy.pyx", "source1.cpp"]
        language="c++",
        extra_compile_args=["-std=c++17"],
   )
]

ext_modules = cythonize(ext_modules)

myclib = ('myclib', {'sources': ["source2.c"]})

setup(
    name="mydummy",
    libraries=[myclib],
    ext_modules=ext_modules,
)

And now building either with

python setup.py build --verbose

or

python setup.py build_clib --verbose build_ext -i --verbose

will first build a simple static library consisting of C-code and link it to the resulting extension consisting of c++-code.

The above code uses the default compilation flags when building the static library, which should be enough in your case.

distutils doesn't offer the possibility to specify additinal flags, so if it is necessary we have either to switch to setuptools which offers this functionality, or to patch the build_clib command.

For the second alternative we have to add the following to the above setup.py-file:

#setup.py
...
# adding cflags to build_clib
from distutils import log
from distutils.command.build_clib import build_clib

# use original implementation but with tweaked build_libraries!
class build_clib_with_cflags(build_clib):
    def build_libraries(self, libraries):
            for (lib_name, build_info) in libraries:
                sources = build_info.get('sources')
                if sources is None or not isinstance(sources, (list, tuple)):
                    raise DistutilsSetupError(
                           "in 'libraries' option (library '%s'), "
                           "'sources' must be present and must be "
                           "a list of source filenames" % lib_name)
                sources = list(sources)

                log.info("building '%s' library", lib_name)


                macros = build_info.get('macros')
                include_dirs = build_info.get('include_dirs')
                cflags = build_info.get('cflags')                    # HERE we add cflags
                objects = self.compiler.compile(sources, 
                                                output_dir=self.build_temp,
                                                macros=macros,
                                                include_dirs=include_dirs,
                                                extra_postargs=cflags,        # HERE we use cflags
                                                debug=self.debug)


                self.compiler.create_static_lib(objects, lib_name,
                                                output_dir=self.build_clib,
                                                debug=self.debug)

...
setup(
    ...
    cmdclass={'build_clib': build_clib_with_cflags}, # use our class instead of built-in!
)

and now we can add additional compile flags to the library-definitions (the previous step can be skipped if setuptools is used):

...
myclib = ('myclib', {'sources': ["source2.c"], 'cflags' : ["-O3"]})
...
ead
  • 32,758
  • 6
  • 90
  • 153
  • I think you misinterpreted a small bit of my answer: I'm suggesting not using `language=c++` in setup.py (it's also already in the original question). Instead I'm suggesting only putting it in the pyx file. The rest of the answer looks like a reasonable way to specify different command-line arguments for different sources – DavidW Dec 17 '19 at 08:42
  • I guess my other concern would be: is the extension linked against `myclib`, or is `myclib` just built? – DavidW Dec 17 '19 at 08:43
  • @DavidW I'm pretty sure that `# distutils: language = c++` has the same effect as setting via `language`-argument of `Extension` (it is not just producing cpp-file from pyx-file but also using g++ for linking). – ead Dec 17 '19 at 09:03
  • @DavidW distutils takes care of passing the built `libmyclib.a` to linker when the actual extensions are linked. – ead Dec 17 '19 at 09:05
  • I'd thought that putting the language in setup.py also forced it on any .c files listed in the `sources` (which wouldn't affect your answer anyway). This appears to be wrong, so I've deleted my answer since this was the main point I was making. I also wasn't sure that `libraries` were linked to everything by default (and there's no documentation). I guess you've tested this so I'll be quiet now... – DavidW Dec 17 '19 at 09:39