0

How to enable python 2.7 library like GDAL in Google App Engine standard? Currently there are linux python-modules in lib-folder in app engine, but when trying to run the code through endpoints, app engine gives internal server error: ImportError: No module named _gdal. I'm using pygdal version 2.2.3.3. Should the libgdal (demanded for pygdal)be installed also on app engine, and if so, how to do it? I installed GDAL locally into lib folder (using ubuntu bash on windows10) following these instructions using this syntax: sudo pip install --target lib --requirement requirements.txt --ignore-installed as it says here. Please help!

samuq
  • 306
  • 6
  • 16

3 Answers3

2

From What compiler can I use to build GDAL/OGR?

GDAL/OGR is written in ANSI C and C++. It can be compiled with all modern C/C++ compilers.

Which means it's incompatible with the (1st generation/python 2.7) standard environment Pure Python sandbox requirement:

All code for the Python runtime environment must be pure Python, and not include any C extensions or other code that must be compiled.

You may want to look at the flexible environment instead. Probably with a custom runtime, see Up-to-date pip with AppEngine Python flex env?

Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
  • Thanks, I'll give that a try. – samuq Nov 19 '18 at 15:01
  • I'm getting error: ImportError: No module named osgeo. I managed to install gdal using following though: https://stackoverflow.com/questions/46348004/gdal-in-flexible-environment/46880451#46880451 How to use that version of gdal in python? I'm currently using: from osgeo import osr – samuq Nov 20 '18 at 13:50
  • Practically the same recipe should be applied to `osgeo` (or any other similar dependencies). You'd have to "merge" the docker building to only have a single dockerfile in the end – Dan Cornilescu Nov 20 '18 at 15:19
1

Google App Engine's standard environment for Python27 only supports a specific set of third-party libraries that use C-extensions, listed here. pygdal is not in the list.

You may want to look into the Python3 standard runtime, though it is in beta. It allows you to install arbitrary third-party libraries.

Andrew F
  • 2,690
  • 1
  • 14
  • 25
  • Thanks for your answer! It seems that pygdal is only for python 2 according to this: https://pypi.org/project/pygdal/ , so what would be equivalent for python 3.7? I've tried everything but installing GDAL in App Engine seems impossible. – samuq Nov 19 '18 at 14:44
  • This *might* work, but only if GDAL is offered as a pip-installable python library, see https://cloud.google.com/appengine/docs/standard/python3/runtime#dependencies – Dan Cornilescu Nov 19 '18 at 14:47
  • @samuq, I didn't realize it was Python2-only. The GAE flexible environment also lets you install arbitrary packages also, but basically by constructing a Docker container. [This](https://stackoverflow.com/a/46880451/6625815) answer seems useful to building such a container. – Andrew F Nov 19 '18 at 15:21
  • @AndrewFiorillo, Thanks, I'll try that solution (flexible environment + Docker container). – samuq Nov 19 '18 at 18:06
0

Modifying this links' answer I managed to get GDAL working in App Engine Flexible. My dockerfile:

FROM gcr.io/google-appengine/python

RUN apt-get update && apt-get -y install libproj-dev libgdal-dev 
RUN export CPLUS_INCLUDE_PATH=/usr/include/gdal
RUN export C_INCLUDE_PATH=/usr/include/gdal
RUN gdal-config --version
# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
RUN virtualenv /env -p python2.7

# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
ADD requirements.txt requirements.txt
RUN pip install -r requirements.txt
# Add the application source code.
ADD . /app

CMD gunicorn -t 120 -b :$PORT main:app

My app.yaml-file:

runtime: custom
env: flex
entrypoint: gunicorn -t 120 -b :$PORT main:app
endpoints_api_service:
    name: xxxxx.com
rollout_strategy: managed
beta_settings:
    cloud_sql_instances: project:europe-north1:dbinstance
runtime_config:
    python_version: 2

requirements.text-file:

pygdal==1.11.3.3
samuq
  • 306
  • 6
  • 16