Why `pyvenv` does not install `python-config`?

Question

I have run into this under MacOS (10.11), but have seen the same problem under various Linuxes as well. I installed the "official" Python3 package, it goes into /Library/Frameworks/Python.framework/Versions/3.4. (Note: the examples below use Python 3.4, but the issue persists with 3.5 as well. I have no access to a machine that has Python 3.6 due to lack of admin privileges, should the issue have been solved in 3.6.)

I need virtual environments and I need the python-config script to figure out which libraries Python3 uses because my project combines Python and C++ code.

If I set up the virtual environment with virtualenv, everything is fine:

$ which virtualenv
/Library/Frameworks/Python.framework/Versions/3.4/bin/virtualenv
$ virtualenv --python=$(which python3) vienv
Running virtualenv with interpreter /Library/Frameworks/Python.framework/Versions/3.4/bin/python3
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.4'
[...blabla...]
Installing setuptools, pip, wheel...done.
$ source vienv/bin/activate
(vienv) $ which python-config
/Users/XXXXX/DEV/STANDALONE/misc/python/vienv/bin/python-config
(vienv) $ python-config --libs
-lpython3.4m -ldl -framework CoreFoundation

However, pyvenv forgets to set up python-config in the virtual environment:

$which pyvenv
/Library/Frameworks/Python.framework/Versions/3.4/bin/pyvenv
$ pyvenv pe
$ source pe/bin/activate
(pe) $ which python-config
/usr/bin/python-config   # !!! Here's the problem !!!
(pe) $ python-config --libs
-lpython2.7 -ldl -framework CoreFoundation

In other words, the system default Python2 python-config stays in my PATH even though I activated the virtual environment.

Now you could say: What's the problem? Use virtualenv and be done with it. However, virtualenv needs to be installed extra via pip and this requires admin rights which I do not always have. pyvenv, OTOH, comes with Python3, or at least that was my understanding.

You could also say: Why don't you just install python-config in your virtual environment using pip? Here's why:

(pe) $ pip install python-config
Requirement already satisfied (use --upgrade to upgrade): python-config in ./pe/lib/python3.4/site-packages
Cleaning up...

Yes, the package is there, but the script itself is not installed into the bin subdirectory of the virtual environment.

Summary: I would like to configure my project so that it can be installed only using the Python3 standard modules/tools, and it does not depend on extra stuff such as virtualenv. And I do not want to pester sysadmins :-)

Question: is there a workaround to get pyvenv install python-config properly? Or: is there another way to figure out which headers and libraries I should use if I link my C++ code against a particular Python3 installation in a virtual environment?

Answer 1

Well, after one year it's time to answer my own question :-)

Here follows the python-config script that is installed by virtualenv into ${VENV}/bin. If you use python3 -m venv ${VENV}, then just copy it into this location manually until this issue gets fixed (NB there's a bug report from 2011 still without a fix as far as I can tell).

#!/usr/bin/env python3

"""
This python-config script was taken from a virtual environment
created by `virtualenv`.
The only change is the hash-bang line.
The user shall copy this to ${VENV}/bin during setup.
:author: unknown + AA
:date: 2018-02-23
"""

import sys
import getopt
import sysconfig

valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags',
              'ldflags', 'help']

if sys.version_info >= (3, 2):
    valid_opts.insert(-1, 'extension-suffix')
    valid_opts.append('abiflags')
if sys.version_info >= (3, 3):
    valid_opts.append('configdir')


def exit_with_usage(code=1):
    sys.stderr.write("Usage: {0} [{1}]\n".format(
        sys.argv[0], '|'.join('--'+opt for opt in valid_opts)))
    sys.exit(code)

try:
    opts, args = getopt.getopt(sys.argv[1:], '', valid_opts)
except getopt.error:
    exit_with_usage()

if not opts:
    exit_with_usage()

pyver = sysconfig.get_config_var('VERSION')
getvar = sysconfig.get_config_var

opt_flags = [flag for (flag, val) in opts]

if '--help' in opt_flags:
    exit_with_usage(code=0)

for opt in opt_flags:
    if opt == '--prefix':
        print(sysconfig.get_config_var('prefix'))

    elif opt == '--exec-prefix':
        print(sysconfig.get_config_var('exec_prefix'))

    elif opt in ('--includes', '--cflags'):
        flags = ['-I' + sysconfig.get_path('include'),
                 '-I' + sysconfig.get_path('platinclude')]
        if opt == '--cflags':
            flags.extend(getvar('CFLAGS').split())
        print(' '.join(flags))

    elif opt in ('--libs', '--ldflags'):
        abiflags = getattr(sys, 'abiflags', '')
        libs = ['-lpython' + pyver + abiflags]
        libs += getvar('LIBS').split()
        libs += getvar('SYSLIBS').split()
        # add the prefix/lib/pythonX.Y/config dir, but only if there is no
        # shared library in prefix/lib/.
        if opt == '--ldflags':
            if not getvar('Py_ENABLE_SHARED'):
                libs.insert(0, '-L' + getvar('LIBPL'))
            if not getvar('PYTHONFRAMEWORK'):
                libs.extend(getvar('LINKFORSHARED').split())
        print(' '.join(libs))

    elif opt == '--extension-suffix':
        ext_suffix = sysconfig.get_config_var('EXT_SUFFIX')
        if ext_suffix is None:
            ext_suffix = sysconfig.get_config_var('SO')
        print(ext_suffix)

    elif opt == '--abiflags':
        if not getattr(sys, 'abiflags', None):
            exit_with_usage()
        print(sys.abiflags)

    elif opt == '--configdir':
        print(sysconfig.get_config_var('LIBPL'))