1

I am trying to compile a simple little hello world example to get a feel for Cython. It works, but the compilation is littered with flags that I do not want. Following the example here I was able to clean up most of the unwanted flags, but a few persist in spite of my best efforts to control the environment variables. The last comment in the linked question indicates that the extra flags might be bakes into the compiler by the vendor, but I have verified that this is unlikely by using several different compilers, all of which default to the same extra flags.

My call to setup.py looks like this:

CC="gcc" CXX="g++" OPT="" CFLAGS="-O3 -D_GNU_SOURCE" BASECFLAGS="" LDFLAGS="" CCSHARED="" LDSHARED="gcc -shared" PY_CORE_FLAGS="" PY_CFLAGS="" AR="" ARFLAGS="" CPPFLAGS="" CPP="" SHLIB_SUFFIX="" python3 setup.py build_ext --inplace

setup.py is very simple:

from setuptools import Extension, setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize([Extension("hellotest", ["hellotest.pyx"])], language_level="3")
)

And finally, the output of gcc during the compilation, with the unwanted flags enclosed in square brackets:

gcc [-DNDEBUG -g -fwrapv -O2 -Wall] -O3 -D_GNU_SOURCE -fPIC -I./ -I. -I/usr/include/python3.6m -c hellotest.c -o build/temp.linux-x86_64-3.6/hellotest.o
gcc [-DNDEBUG -g -fwrapv -O2 -Wall] -O3 -D_GNU_SOURCE -fPIC -I./ -I. -I/usr/include/python3.6m -c ./funcs.c -o build/temp.linux-x86_64-3.6/./funcs.o
gcc -shared -O3 -D_GNU_SOURCE build/temp.linux-x86_64-3.6/hellotest.o build/temp.linux-x86_64-3.6/./funcs.o -o build/lib.linux-x86_64-3.6/hellotest.cpython-36m-x86_64-linux-gnu.so

Does anyone have any idea where they are finding their way into the compiler options?

DavidW
  • 29,336
  • 6
  • 55
  • 86
KBriggs
  • 1,220
  • 2
  • 18
  • 43
  • 1
    I've added a "setuptools" since I'm pretty sure the flags are added by setuptools. I don't know exactly where, but hopefully adding the tag will get the question in front of people that do. – DavidW Apr 14 '20 at 16:46

1 Answers1

2

Setuptools inherits a lot of functionality from distutils and finding out the cflags for building of the c-extensions is one of them.

When build_ext command is run, it calls distutils.sysconfig.customize_compiler(), which in its turn calls distutils.sysconfig.get_config_vars():

def customize_compiler(compiler):
    ...
        (cc, cxx, cflags, ccshared, ldshared, shlib_suffix, ar, ar_flags) = \
            get_config_vars('CC', 'CXX', 'CFLAGS',
                            'CCSHARED', 'LDSHARED', 'SHLIB_SUFFIX', 'AR', 'ARFLAGS')
   ...

to get the default cflags, which will be later extended by the values from the CFLAGS-environment variable and from setup-file and passed to the compiler.

get_config_vars() parses (at least on Linux) the makefile and also config-header, i.e. pyconfig.h, and thus gets the cflags (among other things) with which the current Python-executable was built. The very same cflags are then used to build extensions.

If needed, one can manipulate those built-in flags, as I have shown for example in this SO-post. Whether it is a smart thing to do is another question.

It is a little bit more subtle on Windows, where get_config_vars() doesn't provide that much information - so other tricks are used to build with right options (see for example this SO-post).

ead
  • 32,758
  • 6
  • 90
  • 153
  • I may ask that as another question, then. It seems odd though that -DNDEBUG and -g are both defaults, I would have assumed they conflicted. – KBriggs Apr 14 '20 at 23:05
  • @KBringgs there is no conflict: `-g` means that debugging information is included into the resulting artifact. Including it means the result is bigger (but not really slower) so often it isn't done in the release mode. However I prefer Anaconda-distribution because they have built it with -g: using profiler or looking around with debugger is much easier when this information is at hand. `-DNDEBUG` means that `assert` (C-function) isn't performed which is what one usually wishes in release mode. – ead Apr 15 '20 at 05:50