0



So I have been struggling to set gdal with gcloud. I am working with anaconda although my virtual environment was set with pip as gcloud will use pip through my requirements to recreate my venv. I keep getting gdal-config not found despite all the attempts to set it up. I couldn't find the solution, gcloud also keeps telling that my pip is outdated despite both virtual and system is updated. So if someone can help me out, it would be much appreciated.
Follow the errors below:

gdal-config not found

Step #1: Collecting GDAL==2.4.2 (from -r requirements.txt (line 13))
Step #1:   Downloading https://files.pythonhosted.org/packages/dc/d5/90339b48bdcabc76124eaa058a32d796963a05624eb418ed5ea5af7db9fa/GDAL-2.4.2.tar.gz (564kB)
Step #1:     Complete output from command python setup.py egg_info:
Step #1:     WARNING: numpy not available!  Array support will not be enabled
Step #1:     /opt/python3.6/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
Step #1:       warnings.warn(msg)
Step #1:     running egg_info
Step #1:     creating pip-egg-info/GDAL.egg-info
Step #1:     writing pip-egg-info/GDAL.egg-info/PKG-INFO
Step #1:     writing dependency_links to pip-egg-info/GDAL.egg-info/dependency_links.txt
Step #1:     writing top-level names to pip-egg-info/GDAL.egg-info/top_level.txt
Step #1:     writing manifest file 'pip-egg-info/GDAL.egg-info/SOURCES.txt'
Step #1:     Traceback (most recent call last):
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 151, in fetch_config
Step #1:         p = subprocess.Popen([command, args], stdout=subprocess.PIPE)
Step #1:       File "/opt/python3.6/lib/python3.6/subprocess.py", line 729, in __init__
Step #1:         restore_signals, start_new_session)
Step #1:       File "/opt/python3.6/lib/python3.6/subprocess.py", line 1364, in _execute_child
Step #1:         raise child_exception_type(errno_num, err_msg, err_filename)
Step #1:     FileNotFoundError: [Errno 2] No such file or directory: '../../apps/gdal-config': '../../apps/gdal-config'
Step #1:     
Step #1:     During handling of the above exception, another exception occurred:
Step #1:     
Step #1:     Traceback (most recent call last):
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 237, in get_gdal_config
Step #1:         return fetch_config(option, gdal_config=self.gdal_config)
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 154, in fetch_config
Step #1:         raise gdal_config_error(e)
Step #1:     __main__.gdal_config_error: [Errno 2] No such file or directory: '../../apps/gdal-config': '../../apps/gdal-config'
Step #1:     
Step #1:     During handling of the above exception, another exception occurred:
Step #1:     
Step #1:     Traceback (most recent call last):
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 151, in fetch_config
Step #1:         p = subprocess.Popen([command, args], stdout=subprocess.PIPE)
Step #1:       File "/opt/python3.6/lib/python3.6/subprocess.py", line 729, in __init__
Step #1:         restore_signals, start_new_session)
Step #1:       File "/opt/python3.6/lib/python3.6/subprocess.py", line 1364, in _execute_child
Step #1:         raise child_exception_type(errno_num, err_msg, err_filename)
Step #1:     FileNotFoundError: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
Step #1:     
Step #1:     During handling of the above exception, another exception occurred:
Step #1:     
Step #1:     Traceback (most recent call last):
Step #1:       File "<string>", line 1, in <module>
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 431, in <module>
Step #1:         setup(**setup_kwargs)
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
Step #1:         return distutils.core.setup(**attrs)
Step #1:       File "/opt/python3.6/lib/python3.6/distutils/core.py", line 148, in setup
Step #1:         dist.run_commands()
Step #1:       File "/opt/python3.6/lib/python3.6/distutils/dist.py", line 955, in run_commands
Step #1:         self.run_command(cmd)
Step #1:       File "/opt/python3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
Step #1:         cmd_obj.run()
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 278, in run
Step #1:         self.find_sources()
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 293, in find_sources
Step #1:         mm.run()
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 524, in run
Step #1:         self.add_defaults()
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 560, in add_defaults
Step #1:         sdist.add_defaults(self)
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
Step #1:         self._add_defaults_ext()
Step #1:       File "/env/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
Step #1:         build_ext = self.get_finalized_command('build_ext')
Step #1:       File "/opt/python3.6/lib/python3.6/distutils/cmd.py", line 299, in get_finalized_command
Step #1:         cmd_obj.ensure_finalized()
Step #1:       File "/opt/python3.6/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized
Step #1:         self.finalize_options()
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 287, in finalize_options
Step #1:         self.gdaldir = self.get_gdal_config('prefix')
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 243, in get_gdal_config
Step #1:         return fetch_config(option)
Step #1:       File "/tmp/pip-install-xltlxoe3/GDAL/setup.py", line 154, in fetch_config
Step #1:         raise gdal_config_error(e)
Step #1:     __main__.gdal_config_error: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
Step #1:     
Step #1:     ----------------------------------------
Step #1: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-xltlxoe3/GDAL/
Step #1: You are using pip version 10.0.1, however version 20.0.2 is available.
Step #1: You should consider upgrading via the 'pip install --upgrade pip' command.
Step #1: The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/cloud-builders/docker@sha256:461bb53c226048a2f5eabebe1d8b4367a02d3a484a8cc7455a21377702bbf4f6" failed: step exited with non-zero status: 1
Step #1: 
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ERROR: (gcloud.app.deploy) Cloud build failed. Check logs at https://console.cloud.google.com/cloud-build/builds/dc93f59e-debf-4115-aa3d-d63c7bf6643e?project=1014796457128 Failure status: UNKNOWN: Error Response: [2] Build failed; check build logs for details

#gdal-info
(base) reinhold@reinhold-C930:~$ apt-cache show libgdal20 | grep Version
Version: 2.4.2+dfsg-1~bionic0
Version: 2.2.3+dfsg-2
(base) reinhold@reinhold-C930:~$ gdal-config --version
2.4.2
(base) reinhold@reinhold-C930:~$ gdal-config --libs
-L/usr/lib -lgdal



requirements

astroid==2.3.3
autopep8==1.5
backcall==0.1.0
blinker==1.4
bokeh==1.3.4
click==7.1.1
decorator==4.4.2
dominate==2.5.1
Flask==1.1.1
Flask-Bootstrap==3.3.7.1
Flask-Mail==0.9.1
Flask-WTF==0.14.3
GDAL==2.4.2
gunicorn==20.0.4
ipykernel==5.1.4
ipython==7.13.0
ipython-genutils==0.2.0
isort==4.3.21
itsdangerous==1.1.0
jedi==0.16.0
Jinja2==2.11.1
jupyter-client==6.0.0
jupyter-core==4.6.3
lazy-object-proxy==1.4.3
MarkupSafe==1.1.1
mccabe==0.6.1
numpy==1.18.2
packaging==20.3
pandas==1.0.2
parso==0.6.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.0.0
prompt-toolkit==3.0.4
ptyprocess==0.6.0
pycodestyle==2.5.0
Pygments==2.6.1
pylint==2.4.4
pyparsing==2.4.6
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3
pyzmq==19.0.0
six==1.14.0
tornado==6.0.4
traitlets==4.3.3
typed-ast==1.4.1
visitor==0.1.3
wcwidth==0.1.8
Werkzeug==1.0.0
wrapt==1.11.2
WTForms==2.2.1
ReinholdN
  • 526
  • 5
  • 22
  • please paste text e.g. requirements.txt into your question rather than images. Images are less easily to read and make it difficult to repro your issue; I'm unwilling to recreate your requirements.txt to try to repro this. – DazWilkin Mar 28 '20 at 18:00
  • alright! I'm doing that now – ReinholdN Mar 28 '20 at 18:19
  • 1
    Thanks. I'm unclear. IIUC you're trying to deploy to App Engine flex using the python3 runtime, correct? I think you can't do it this way because of the non-python dependencies required by GDAL. Have you seen this: https://stackoverflow.com/a/46880451/609290 ? – DazWilkin Mar 28 '20 at 19:07
  • Exactly. https://storage.cloud.google.com/staging.raphaelnogueira.appspot.com/eu.gcr.io/raphaelnogueira/appengine/default.20200328t104413%3Alatest – ReinholdN Mar 28 '20 at 19:10
  • Thanks @DazWilkin I need a create a docker container, then customize my gunicorn to it, replace django for flask and hopefully it will work. Thanks! I will have to grasp the dockerfile concept first. – ReinholdN Mar 28 '20 at 19:50
  • 1
    Yes, I think that's the correct way to go. If you're unfamiliar with Docker(file), I recommend you start with the `Dockerfile` from that answer as your basis. Think of the Dockerfile as mirroring your install script and that should be a good foundation – DazWilkin Mar 28 '20 at 20:12
  • @DazWilkin I much appreciate your help man! Really! – ReinholdN Mar 28 '20 at 20:38
  • Sorry I couldn't be more help! I'm very familiar with GCP but unfamiliar with GDAL. I tried variousn combo's of Dockerfile mirroring the other answer but I'm unable to get GDAL to compile when it's pip installed. Hopefully someone else is able to help! – DazWilkin Mar 28 '20 at 21:15
  • You gave me the north mate @DazWilkin that's all I needed., I am currently working on the solution. – ReinholdN Mar 29 '20 at 09:23
  • Hi @DazWilkin, how are you doing man? So I did get docker running, understood the concepts and try to deploy following your instructions, I got stuck in an error, I checked few forums and I could not get it fixed, I thought you would be able to help me, as it seems your area of expertise. I am getting this error. /bin/sh: 1: pip: not found – ReinholdN Apr 18 '20 at 10:14
  • here is my dockerfile ``` FROM gcr.io/google-appengine/python RUN apt-get update && apt-get install -y \ python3-pip \ binutils \ gdal-bin \ python-gdal RUN virtualenv /venv_conda -p python3.7 ENV VIRTUAL_ENV /venv_conda ENV PATH /home/reinhold/anaconda3/envs/venv_conda/lib/python3.7/site-packages ADD requirements.txt requirements.txt RUN pip install -r requirements.txt ADD . /app CMD gunicorn -b :$PORT wsgi.py ``` – ReinholdN Apr 18 '20 at 10:16
  • Apparently I should open a new question to be able to share all the details with you, in case you cant figure it out let me know then I will update the github and share it with you. I was using a pip virtual env, though it was using python 2.7 instead 3.7, after good reading I found out the best would use anaconda as I am running python through anaconda. Also I havent been in quarantine, so life still busy for me. Thanks – ReinholdN Apr 18 '20 at 10:19
  • https://github.com/Reinhold83/IHM.git – ReinholdN Apr 18 '20 at 10:30
  • Thanks for the update! Glad your issue is resolved. – DazWilkin Apr 18 '20 at 14:11
  • I still stuck though! Step 7/9 : RUN pip install -r requirements.txt ---> Running in b5358936867c /bin/sh: 1: pip: not found The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 127 – ReinholdN Apr 18 '20 at 14:40

0 Answers0