For a project shipping with a pre-built customized Python distribution I need to be able to compile packages from source using pip (within the installed environment).
This is what the file system structure for two installations of the final product might look like:
/opt
├── my-program-v1
│ ├── some-files
│ ├── custom-python-3.9
├── my-program-v2
│ ├── some-files
│ ├── custom-python-3.11
Since the readily installed program (together with its Python installation) might be installed to any directory, pip needs a way to find the header files for the used Python installation.
Just copying the whole Python installation to the desired directory will result in errors when trying to pip install a package that needs to be built from source (e.g. ibm_db):
$ python3 -m pip install ibm_db
Collecting ibm_db
Downloading ibm_db-3.1.4.tar.gz (1.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 9.5 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Detected 64-bit Python
Detected platform = linux, uname = x86_64
Downloading https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/linuxx64_odbc_cli.tar.gz
Downloading DSDriver from url = https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/linuxx64_odbc_cli.tar.gz
Pre-requisite check [gcc] : Passed
No Python.h header file detected.
Please install python-devel or python3-devel as instructed in "https://github.com/ibmdb/python-ibmdb/blob/master/README.md#KnownIssues" and continue with the installation
[end of output]
Usually, when you build Python from source you would set the final destination folder using configure --prefix=/opt/my-program-v2, which would be stored inside the sysconfigdata of that readily built Python installation.
But how can this be done when you need to pre-build Python and don't know the installation directory in advance?
One possible way is to manually modify the generated sysconfig-data, (usually found in lib/<python-version>/_sysconfigdata__linux_x86_64-linux-gnu.py) but this is not fun (due to the used format) and introduces a step outside of the CI.
How do others do that? Is there an intended way to place/relocate a prebuilt installation of python?
