1

I currently use 'setuptools' to automatically cythonize and compile my Cython modules on Linux using gcc. From now on, I need more control over the build flags supplied to gcc. If I use the following in my setup.py:

cythonize(
    [Extension("*", ["project/*.pyx"])
    nthreads=4
)

I get build flags, that look like:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -fPIC -I./fastmat/core -I/home/seb/.local/lib/python3.6/site-packages/numpy/core/include -Iproject/core -Ifastmat/inspect -Iutil -I/usr/include/python3.6m -c project/BlockDiag.c -o build/temp.linux-x86_64-3.6/project/BlockDiag.o

Here I am totally flabbergasted by the fact that several build flags occur multiple times and without issuing this in any (to me obvious) way.

How can I clean up these build flags, such that they look like the ones suggested here? I hope to learn something about setuptools along the way to ultimately get full control over the build process without having to use a self-maintained makefile.

DavidW
  • 29,336
  • 6
  • 55
  • 86
Labello
  • 323
  • 3
  • 15
  • For anyone interested in 32-bit cross-compilation: https://stackoverflow.com/a/4201282/1959808, https://unix.stackexchange.com/a/352784/43390, https://askubuntu.com/a/454254/173666, https://askubuntu.com/a/905083/173666, https://stackoverflow.com/a/23227562/1959808 – 0 _ Jun 30 '21 at 12:57

1 Answers1

1

The flags GCC gets come from one of the env variables. Enter

$ python -c "from distutils import sysconfig;\
print(sysconfig.get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',\
'BASECFLAGS', 'LDFLAGS', 'CCSHARED', 'LDSHARED', 'SO'))"

to print them. This is what distutils uses by default for extension compilation. Now check which env var introduces which flag and override the env vars accordingly, for example

$ CC="gcc-7.3.0" CFLAGS="-Ofast" python setup.py build_ext

to use the specific compiler version and turn on O3 optimizations.

Also, it looks like you're using numpy.distutils instead of vanilla distutils, so be aware of extra include/link flags numpy adds under the hood.

hoefling
  • 59,418
  • 12
  • 147
  • 194
  • Thanks! I got it cleaned up to `gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./fastmat/core [...] -o build/temp.linux-x86_64-3.6/fastmat/BlockDiag.o` by using `CC="gcc" CXX="g++" OPT="" CFLAGS="" BASECFLAGS="" LDFLAGS="" CCSHARED="" LDSHARED="gcc -shared" PY_CORE_CFLAGS="" PY_CFLAGS="" SO="" python setup.py build_ext --inplace`. Now I am still wondering where the remaining flags come from, because I searched the dictionary, which is returned by `sysconfig.get_config_vars()` for any key containing `-O3` for example and set it to `""`, but still it continues to pop up. – Labello Jun 28 '18 at 09:21
  • 1
    And for clarification, I am just linking against some numpy libs, but I am using the standard distutils lib. – Labello Jun 28 '18 at 09:23
  • 1
    If you can't find the flags being set in one of the env vars (for reference, [here's the spot in `distutils` source where the flags are read from env](https://github.com/python/cpython/blob/master/Lib/distutils/sysconfig.py#L173)), then they're probably baked into `gcc` by the vendor or distro maintainer (you can check `gcc -dumpspecs` for that). If you need a vanilla `gcc`, build one from source and pass the custom executable via `CC` var. – hoefling Jun 28 '18 at 10:44