1

I am trying to install python-snappy in Google Cloud Composer through Google Cloud Platform console. I am using python3 environment.

I had tried to install different python-snappy versions (0.5.3, 0.5.2, 0.5.1, 0.5, 0.4), however, it always raises this error:

Http error status code: 400 Http error message: BAD REQUEST Additional errors: {"ResourceType":"w071b50175ee0733c-tp/asia-northeast1-kumparan-co-e2ca75b3-gae-typer:appengine.apps.services.versions.create","ResourceErrorCode":"400","ResourceErrorMessage":"Docker image gcr.io/w071b50175ee0733c-tp/c949bad3-ca30-4717-a975-8e36bae3929a was either not found, or is not in Docker V2 format. Please visit https://cloud.google.com/container-registry/docs/ui "}

I also had tried to rerun, as discussed here: https://groups.google.com/forum/#!topic/cloud-composer-discuss/jpxAGCPFkZo.

But, it doesn't solve this issue in my case.

Any body knows how to solve it?

Sugimiyanto
  • 320
  • 3
  • 18
  • Can you include the exact install command you used? – Dustin Ingram Jan 14 '19 at 18:33
  • as I said, I am using GCP console to install the library. So, it is done just by inserting the library name. You can take a look at this link, see `Installing a Python dependency from PyPi` part in tab `console`. https://cloud.google.com/composer/docs/how-to/using/installing-python-dependencies – Sugimiyanto Jan 15 '19 at 03:37

1 Answers1

3

If you navigate to https://console.cloud.google.com/cloud-build/builds/, you should see a failed build with the following logs:

Failed building wheel for python-snappy
Running setup.py clean for python-snappy
Failed to build python-snappy
Installing collected packages: python-snappy
Running setup.py install for python-snappy: started
Running setup.py install for python-snappy: finished with status 'error'
Complete output from command /usr/local/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-M5s4qk/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-EuqCf8/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy_cffi_builder.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy_cffi.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/__main__.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy_formats.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/hadoop_snappy.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/__init__.py -> build/lib.linux-x86_64-2.7/snappy
running build_ext
building 'snappy._snappy' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/snappy
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python2.7 -c snappy/snappymodule.cc -o build/temp.linux-x86_64-2.7/snappy/snappymodule.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
snappy/snappymodule.cc:31:22: fatal error: snappy-c.h: No such file or directory
#include <snappy-c.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1

This is because the python-snappy package is only the Python bindings to the underlying C library, not the C library itself.

See "Error installing python-snappy: snappy-c.h: No such file or directory" for more details.

Dustin Ingram
  • 20,502
  • 7
  • 59
  • 82
  • yes, you're right. the problem here is we cannot install C library in Google Cloud Composer, because C library is a system-level package. Google Cloud Composer doesn't support installing arbitrary system packages, as discussed here: https://groups.google.com/forum/#!topic/cloud-composer-discuss/jpxAGCPFkZo – Sugimiyanto Jan 16 '19 at 06:39