2

I'm trying to add unixODBC as a vendor app dependency, as per notes that are outlined here: https://github.com/cloudfoundry/python-buildpack/issues/34, but I'm still getting fatal error:

sql.h: No such file or directory.

I've created a Minimal, Verifiable, Complete script below to facilitate readers with reproducing my issue:

mkdir new_app
cd new_app

cat <<EOF>requirements.txt
pyodbc==4.0.17
EOF

cat <<EOF>manifest.yml
applications:
 - name: pyodbc-test
   memory: 128M
EOF

cat <<EOF>.profile
export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/app/vendor/unixodbc/lib
EOF

sudo docker run -it --rm -v ${PWD}:/app cloudfoundry/cflinuxfs2 bash -c \
   "wget ftp://ftp.unixodbc.org/pub/unixODBC/unixODBC-2.3.4.tar.gz;  tar xzf unixODBC-2.3.4.tar.gz; cd unixODBC-2.3.4;  ./configure --prefix=/app/vendor/unixodbc; make; make install;"

Followed by:

sudo chown -R ${USER}:${USER} vendor/
pip3 download -r requirements.txt -d vendor/
cf push

The cf push output:

...
-------> Buildpack version 1.5.15
 !     Warning: Your application is missing a Procfile. This file tells Cloud Foundry how to run your application.
 !     Learn more: https://docs.cloudfoundry.org/buildpacks/prod-server.html#procfile
-----> Installing python-2.7.13
Downloaded [file:///tmp/buildpack/dependencies/https___buildpacks.cloudfoundry.org_dependencies_python_python-2.7.13-linux-x64.tgz]
     $ pip install -r requirements.txt
       Collecting pyodbc==4.0.17 (from -r requirements.txt (line 1))
       Installing collected packages: pyodbc
         Running setup.py install for pyodbc: started
           Running setup.py install for pyodbc: finished with status 'error'
           Complete output from command /app/.heroku/python/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-sxxn23/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-lRnuMu-record/install-record.txt --single-version-externally-managed --compile:
           running build
           running build_ext
           building 'pyodbc' extension
           creating build/temp.linux-x86_64-2.7
           creating build/temp.linux-x86_64-2.7/src
           gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPYODBC_VERSION=4.0.17 -I/app/.heroku/python/include/python2.7 -c src/errors.cpp -o build/temp.linux-x86_64-2.7/src/errors.o -Wno-write-strings
           cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
           In file included from src/errors.cpp:2:0:
           src/pyodbc.h:56:17: fatal error: sql.h: No such file or directory
            #include <sql.h>
                            ^
           compilation terminated.

           ----------------------------------------
           running install
           creating build
           error: command 'gcc' failed with exit status 1
Command "/app/.heroku/python/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-sxxn23/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-lRnuMu-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-sxxn23/pyodbc/
Failed to compile droplet
Exit status 223
Staging failed: Exited with status 223
Destroying container
Successfully destroyed container

There is a similar question here: pyodbc install on IBM Bluemix server error, however that question is different to this question in that it is very generic, whereas this question is on the same topic but relates to a very specific set of build steps.

Update: Here is the updated script based on the accepted answer:

mkdir new_app
cd new_app

cat <<EOF>environment.yml
name: pyodbc-test
dependencies:
- pyodbc
EOF

cat <<EOF>manifest.yml
applications:
 - name: pyodbc-test
   memory: 128M
EOF

cf push
Chris Snow
  • 23,813
  • 35
  • 144
  • 309

2 Answers2

4

src/pyodbc.h:56:17: fatal error: sql.h: No such file or directory #include

Your setup generally looks OK, but when pip runs during staging the build tools are still unable to find the library headers. The headers are there since you're including them in the vendor/ directory, but it doesn't know they're there as the build script for the python module won't look for them in that location.

I don't know of a way to make it look for the headers in a non-standard location without modifying the build pack because you need to send additional arguments to pip.

A few people suggested setting the CPATH and/or C_INCLUDE_PATH environment variables as these are supposed to get picked up by gcc, but in my tests, these were ignored.

In the end, I think your best bet is to use Conda to install pyodbc. Here's how you'd go about that.

  1. Add environment.yml to the root of your project.
  2. In it add the following:
name: pyodbc-test
dependencies:
- pyodbc
  1. Run cf push.

The build pack should run during staging, install miniconda and then install your dependencies with conda, specifically pyodbc.

Hope that helps!

Daniel Mikusa
  • 13,716
  • 1
  • 22
  • 28
2

As per @Daniel Mikusa's answer...

you need to send additional arguments to pip.

Pip supports --global-options to pass this sort of thing along. Unfortunately, from here the options are passed to setup.py.

setup.py has it's own option for globals which is build_ext... 

Options for 'build_ext' command:
  --build-lib (-b)     directory for compiled extension modules
  --build-temp (-t)    directory for temporary files (build by-products)
  --plat-name (-p)     platform name to cross-compile for, if supported
                       (default: linux-x86_64)
  --inplace (-i)       ignore build-lib and put compiled extensions into the
                       source directory alongside your pure Python modules
  --include-dirs (-I)  list of directories to search for header files
  ...

Setup.py needs build_ext --include-dirs <path to unixodbc/include>

As far as I'm aware, to give those arguments to setup.py each argument must be individually prepended by --global-option=.

I just went through the same headache, and the following worked for me.

pip install pyodbc --global-option="build_ext" --global-option="--include-dirs" --global-option="<path to unixodbc/include>"
jayreed1
  • 152
  • 2
  • 8