8

For my python extension, I have both C (from an embedded library) and C++ files and they are compiled and linked together. Only the C++ part interfaces with Python (via SWIG). This works both in windows with VS2015 and gcc under linux. However, with gcc, the C++ files need a different set of compiler flags (eg. -std=c++11, -Wno-reorder) than the C files, to avoid warnings concerning the improper flags in C.

Is there a way in setuptools / distutils to change compiler flags for each file seperatly, eg. based on the file extension?

I use already a custom build step from https://stackoverflow.com/a/36293331/3032680.

Update:

The main problem is, that distutils.ccompiler does not check the file extension for C or C++ and is running everything with $CC. Even defining CXXFLAGS does not help. I will stand the warnings, neither with export nor a definition using os.eniviron in the setup.py file.

Update 2:

On macOS with CLang 8.0.0 the situation gets worse: Trying to compile a .c file with -std=c++11 is not a warning but an error.

oekopez
  • 1,080
  • 1
  • 11
  • 19
  • This question is a duplicate of [How do I specify different compiler flags for just one Python/C extension source file?](https://stackoverflow.com/q/15527611/11725753) – EvgenKo423 Jul 24 '21 at 15:03
  • This question asks for extensions with mixed C and C++ code files, while the other question is not about mixing the two languages in one extension - hence the answers do not help with that problem. – oekopez Jul 26 '21 at 07:28
  • I've read your whole question, not just title. While your main problem is how to get rid of warnings/errors, the question you were asking here is how "to change compiler flags for each file seperatly". Surely the answers from another question can't solve your problem directly, but they can with a minor modification as the only difference is condition. – EvgenKo423 Jul 26 '21 at 07:48
  • I've done quite some research before I posted this question 3 years ago, but obviously I did not found the, even older, Q you are linking here. My historic self (or a recent other person in the same situation) would have profitted from this question and the two answers (including my own, part two, which is not an answer to the "duplicate" question, but only to this) - a link to this other question is helpful, too. Closing the Q as duplicate makes the content more difficult to find. – oekopez Jul 26 '21 at 08:08
  • Surely your different wording is useful, but it's not a penalty for you – it's a guidance for other people with the same problem. That's what duplicates are for, they actually make it _easier_ to find by putting a big bold message at the top and preventing people from writing answers in multiple places. It's not gonna be deleted. This comment also adds a back-link to the other question. ;-) – EvgenKo423 Jul 26 '21 at 08:34

2 Answers2

1

There is another option which consists in overloading the distutils compiler class (say the unix C compiler):

import os
from distutils.unixccompiler import UnixCCompiler

cpp_flags = ['-std=c++11']

class C_CxxCompiler(UnixCCompiler):
  def _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts):
    _cc_args = cc_args

    # add the C++ flags for source files with extensions listed below
    if os.path.splitext(src)[-1] in ('.cpp', '.cxx', '.cc'):
      _cc_args = cc_args + cpp_flags

    UnixCCompiler._compile(self, obj, src, ext, _cc_args, extra_postargs, pp_opts)

Then you overload the distutils.build_ext command to intercept the extension building and replace the compiler before the compilation proceeds:

class BuildC_CxxExtensions(build_ext):
  def build_extensions(self, ext):
    if self.compiler.compiler_type == 'unix':
      # Replace the compiler
      old_compiler = self.compiler
      self.compiler = C_CxxCompiler()

      # Copy its attributes
      for attr, value in old_compiler.__dict__.items():
        setattr(self.compiler, attr, value)
    build_ext.build_extensions(self, ext)

Depending on your platform, you might need to overload the other compiler classes MSVCCompiler, CygwinCCompiler, Mingw32CCompiler or BCPPCompiler.

bcoconni
  • 11
  • 2
  • Note that: 1. `cc_args` is a list, so your code will return an error; 2. It's undesirable to prepend flags, because they can be overridden by default ones; 3. This method will work only for Unix compilers (and their ports), because others don't use the `_compile()` method. – EvgenKo423 Jul 24 '21 at 17:04
  • @EvgenKo423: 1. Correct, I've modified `cpp_flags` to be a list; 2. OK I've modified the order of the operations in `_cc_args = cpp_flags + cc_args`; 3. Most likely true (I've not tried myself), however the OP specifically said that he hit the problem with `gcc` i.e with `UnixCompiler`. – bcoconni Jul 25 '21 at 11:27
  • I found this helpful, thanks. The only thing I had to change was use `def build_extension` instead of `def build_extensions` (not plural). `build_extension` takes an `ext` argument while `build_extensions` takes no extra arguments. – Tails86 Sep 01 '23 at 21:51
0

Since distutils goes a long way to ensure that all files get compiled with the same compiler flags regardless of their file extension .c or .cpp. Hence even using CFLAGS and CXXFLAGS are not taken into account but gcc and CLang still handles them differently. Visual Studio just compiles everything as C++.

I got my problem resolved by accepting that C is in most cases still a subset of C++ and renamed the C-Source files to .cpp, even if the files contain C. This solution is ugly but I got rid of the warnings in gcc and the errors with CLang - especially since this solution blurs again the language barrier between C and C++.

A second solution I adopted later is to create a static library from the C Code outside distutlis and link the Python extension to that static library.

oekopez
  • 1,080
  • 1
  • 11
  • 19