Unable to deploy GIS due gdal-config not found

Question

I have been trying to deploy GIS data for a good while at this point with gcloud in the google app engine, I always will stop at the same error no matter how many different approaches I attempted. With dockfiler, without it, with pygdal, it will come to a point where it won't be able to find the gdal-config, i.e., the headers. Both system and virtualenv have the same versions.

gdal-config --libs -L/usr/lib -lgdal gdal-config --version 2.4.2 pip freeze|grep pygdal 2.4.2.5 libgdal 2.4.2+dfsg-1~bionic0
System ubuntu 18.0.4
I am using VScode

Here is my app.yaml

runtime: python
env: flex
entrypoint: gunicorn -b :$PORT app:app

runtime_config:
    python_version: 3

wsgi.py file

from app import app

if __name__ == "__main__":
    app.run(host='127.0.0.1', port=8080, debug=True)

launch.json file

{
        {
            "name": "Python: Flask",
            "type": "python",
            "request": "launch",
            "module": "flask",
            "env": {
                "FLASK_APP": "app.py",
                "FLASK_ENV": "development",
                "FLASK_DEBUG": "0"
            },
            "args": [
                "run",
                "--no-debugger",
                "--no-reload"
            ],
            "jinja": true
        }
    ]
}

requirements.txt file

astroid==2.3.3
attrs==19.3.0
autopep8==1.5.2
blinker==1.4
bokeh==1.3.4
click==7.1.1
click-plugins==1.1.1
cligj==0.5.0
dominate==2.5.1
Fiona==1.8.13.post1
Flask==1.1.2
Flask-Bootstrap==3.3.7.1
Flask-Mail==0.9.1
geopandas==0.7.0
gunicorn==20.0.4
isort==4.3.21
itsdangerous==1.1.0
Jinja2==2.11.2
lazy-object-proxy==1.4.3
MarkupSafe==1.1.1
mccabe==0.6.1
munch==2.5.0
numpy==1.18.3
packaging==20.3
pandas==1.0.3
Pillow==7.1.1
pycodestyle==2.5.0
pygdal==2.4.2.5
pylint==2.4.4
pyparsing==2.4.7
pyproj==2.6.0
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3.1
Shapely==1.7.0
six==1.14.0
tornado==6.0.4
typed-ast==1.4.1
visitor==0.1.3
Werkzeug==1.0.1
wrapt==1.11.2
wtf==0.1

Error

Step #1: Collecting pygdal==2.4.2.5 (from -r requirements.txt (line 29))
Step #1:   Downloading https://files.pythonhosted.org/packages/59/2b/44f117abcf855a7bd6b8aa4432036d94229a2c6309277c3538def9e0568b/pygdal-2.4.2.5.tar.gz (446kB)
Step #1:     Complete output from command python setup.py egg_info:
Step #1:     running egg_info
Step #1:     creating pip-egg-info/pygdal.egg-info
Step #1:     writing pip-egg-info/pygdal.egg-info/PKG-INFO
Step #1:     writing dependency_links to pip-egg-info/pygdal.egg-info/dependency_links.txt
Step #1:     writing requirements to pip-egg-info/pygdal.egg-info/requires.txt
Step #1:     writing top-level names to pip-egg-info/pygdal.egg-info/top_level.txt
Step #1:     writing manifest file 'pip-egg-info/pygdal.egg-info/SOURCES.txt'
Step #1:     error: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
Step #1:     
Step #1:     ----------------------------------------
Step #1: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-8gfn0078/pygdal/
Step #1: You are using pip version 10.0.1, however version 20.1b1 is available.
Step #1: You should consider upgrading via the 'pip install --upgrade pip' command.
Step #1: The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/cloud-builders/docker@sha256:461bb53c226048a2f5eabebe1d8b4367a02d3a484a8cc7455a21377702bbf4f6" failed: step exited with non-zero status: 1
Step #1: 
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ERROR: (gcloud.app.deploy) Cloud build failed. Check logs at https://console.cloud.google.com/cloud-build/builds/c8d993a9-c897-462f-a68c-ca0a4b97e35d?project=791950161693 Failure status: UNKNOWN: Error Response: [2] Build failed; check build logs for details

Thanks beforehand, this is annoying to an extreme at this point.

score 0 · Answer 1 · answered Apr 24 '20 at 11:33

0

This seems to be an issue with GDAL that has been changing but not fixed for a long time now.

Please check the following links that mention different workaround to get past this issue. First, I recommend you get it working locally, then dockerize it with all the steps needed to bypass this issue, and finally try to deploy it to GAE Flex.

Error: gdal-config not found

https://docs.djangoproject.com/en/3.0/ref/contrib/gis/install/geolibs/

https://gis.stackexchange.com/questions/28966/python-gdal-package-missing-header-file-when-installing-via-pip

https://github.com/shaharkadmiel/pySW4/issues/16

answered Apr 24 '20 at 11:33

Waelmas

1,894
1
9
19

Thanks @Waelmas, I had read all of them expected the djangoproject before and tried all the approaches. I am using anaconda, so I also tried a virtualenv with anaconda as well, still I didnt get it running. I will keep trying, though I started to wonder if there an alternative library for GDAL. Do you have any recommendations? Locally my webapp runs just fine, the issue it when I try to deploy it through gcloud. I saw few posts where they use .yaml file to set install gdal too. – ReinholdN Apr 25 '20 at 15:21
I'm not sure about any good alternatives to recommend. However, regarding installing it in a different way, I suggest that you add all the steps needed in your Dockerfile just like the steps used to prepare the environment locally. This way it will behave just like you have it locally. (which means creating your custom runtime env for GAE Flex) – Waelmas Apr 27 '20 at 12:11
thanks again for your time and help. I will work on my dockerfile, I downloaded gdal guide line as well. I quite confident this time will work. – ReinholdN Apr 27 '20 at 12:47

Unable to deploy GIS due gdal-config not found

1 Answers1

Linked