7

Maybe a stupid question, but I was wondering where Python's distutils get the compiler options from? It gets some linked directories wrong and I want to correct that once and for all.

I know there should be a prefix/lib/pythonver/distutils/distutils.cfg but I can't find any distutils.cfg anywhere on the computer. Obviously I haven't got a local setup.cfg or any $HOME/.pydistutils.cfg.

I'm using the Enthought 64-bit distribution, version 7.3 (Python 2.7) on Mac OS X 10.8.3

Cheers, U.

Mulle
  • 73
  • 1
  • 3

2 Answers2

7

I actually export them to the environment, just like for autotools' configure:

export CC=/usr/local/bin/clang
export CFLAGS=-I${HOME}/include
export LDFLAGS=-lboost

If you also need to override the linker separately:

export LDSHARED=/usr/local/bin/clang -shared

And if you don't like exporting the settings to your environment, do something like this for a one-time setting:

CC=/usr/local/bin/clang CFLAGS=-I${HOME}/include python setup.py build

If you want to find out what the default options were when python was build, use python-config --<flag>. Some flags are cflags, ldflags, libs or includes.

kynan
  • 13,235
  • 6
  • 79
  • 81
  • Alright, that would set the variables, but they aren't set right now, so where does distutils get them from? I haven't found any hint in the python files that distutils would read it out somewhere else. – Mulle Apr 18 '13 at 15:38
  • See also Eric's answer; if there's no config file (you say it isn't there), the compiler flags indeed follow the original configure/make flags, and thus can also be seen with `python-config --cflags` etc. –  Apr 18 '13 at 17:23
  • Ah, there they are! Since I haven't compiled Python but took the EPD distribution I wouldn't have the Makefile I guess. But now I know where to look and what to do. Cheers! – Mulle Apr 19 '13 at 17:34
  • Finally got enough time to try the above and it sort of works. I included the correct paths and options into my .bashrc in the form `export CFGLAGS='...loads of options and paths...'`. However, distutils now gets both sets of flags and some of them contradict each other, such as -O3 and -O2. Programme compiles fine though... – Mulle Apr 22 '13 at 07:46
  • @Mulle: if I have it correctly, for `-L` and `-I` options, they are interpreted in the order of appearance. So if your local include directory is specified before the default ones (which I think happens when using `CFLAGS and `LDFLAGS`), your local include files get picked up first. For the `-O` options, however, the `gcc` manual page says: "If you use multiple -O options, with or without level numbers, the last such option is the one that is effective.". If that works incorrectly for you, you could use `-fno-...` options to turn off optimisations. –  Apr 22 '13 at 08:17
  • See also [this question](http://stackoverflow.com/questions/6928110/how-may-i-override-the-compiler-gcc-flags-that-setup-py-uses-by-default). –  Apr 22 '13 at 10:09
  • you may want to try `export CPATH=$(env | grep _INC | cut -d= -f2 | paste -d: -s)` and `export LIBRARY_PATH=$(env | grep _LIB | cut -d= -f2 | paste -d: -s)` – George Dec 01 '17 at 18:24
  • On my system default values are stored in `/usr/lib/python3.7/_sysconfigdata_m_linux_x86_64-linux-gnu.py`. And actually `python-config --cflags` is original `CFLAGS` + `python-config --includes`. So, ideally I've got to take `CFLAGS` from `_sysconfigdata_`-file, add my custom options, and pass all that to `distuils` (`CFLAGS=... pip install ...`). Isn't there an easier way (from the user's standpoint)? – x-yuri Aug 16 '18 at 09:55
  • Well, the better way is probably using [`--global-option` thing](https://stackoverflow.com/a/22942120/52499). Additionally, `--global-option` disables reusing already built [wheel](https://github.com/pypa/pip/blob/18.0/src/pip/_internal/cmdoptions.py#L58-L59). With env vars, you've got to add `--no-cache-dir`, or delete the wheel, for your env vars to take effect. E.g., `find ~/.cache/pip/wheels -name 'cryptography-*.whl' -delete` – x-yuri Aug 16 '18 at 11:52
3

Compiler options are taken from CPython’s Makefile. IOW they are the same as the ones used to compile Python. You can override most of them on the command line as Evert described.

The global distutils.cfg is something that a sysadmin can create to set default options, not a file that is installed with Python.

merwok
  • 6,779
  • 1
  • 28
  • 42