10

There's a similar question that was recently responded to on Stackoverflow here: Google Cloud Storage Client not working on dev appserver

The solution was to either upgrade the SDK to 1.8.8 or use the previous revision of the GCS client library which didn't have the bug instead.

I'm currently using 1.8.8 and have tried downloading multiple revisions and /_ah/gcs doesn't load for me. After using up a significant number of my backend instances trying to understand how GCS and app engine work together, it'd be great if I could just test it on my local server instead!

When I visit localhost:port/_ah/gcs I get a 404 not found error.

Just a heads up, to install the library all I did was drag and drop the code into my app folder. I'm wondering if maybe I skipped a setup step? I wasn't able to find the answer in the documentation!

thanks!!

Note To clarify this is my first week using GCS, so my first time trying to use the dev_server to host it.

Community
  • 1
  • 1
iceanfire
  • 563
  • 6
  • 15
  • 6
    hey, did you ever figure this out? I'm stuck on this, too. – Andy Dennie Jan 22 '14 at 22:20
  • nope, i've moved that part of the processing to out of appengine--the restrictions + inability to test in the dev environment were killing my development time! Still use appengine for most of our code though, and it's amazing but expensive (we're trying to also cut costs and use it more efficiently). – iceanfire Jan 23 '14 at 11:54
  • 5
    Compared to other solutions, e.g. heroku, Google Cloud seems like a big mess and is horribly documented! – electronix384128 Jul 06 '16 at 22:28
  • Have you found the solution? – Ajeet Oct 25 '17 at 12:02

3 Answers3

4

I was able to find the google cloud storage files I wrote to a bucket locally at:

    localhost:port/_ah/gcs/bucket_name/file_suffix

Where port is by default 8080, and the file was written to: /bucket_name/file_suffix

For those trying to understand the full process of setting up a simple python GAE app and testing local writes to google cloud storage:

1. Follow the google app engine "quickstart":

https://cloud.google.com/appengine/docs/standard/python/quickstart

2. Run a local dev server with:

    dev_appserver.py app.yaml 

3. If using python, follow "App Engine and Google Cloud Storage Sample":

https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/app-engine-cloud-storage-sample

If you run into "ImportError: No module named cloudstorage" you need to create a file named appengine_config.py

    touch appengine_config.py

and add to it:

    from google.appengine.ext import vendor
    vendor.add('lib')

GAE runs this script automatically when starting your local dev server with dev_appserver.py app.yaml, and it is necessary to run this script for GAE to find the cloudstorage library in your lib/ folder

4. "Writing a file to cloud storage" from the same tutorial:

    def create_file(self, filename):
    """Create a file."""

       self.response.write('Creating file {}\n'.format(filename))

       # The retry_params specified in the open call will override the default
       # retry params for this particular file handle.
       write_retry_params = cloudstorage.RetryParams(backoff_factor=1.1)
       with cloudstorage.open(
           filename, 'w', content_type='text/plain', options={
               'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
               retry_params=write_retry_params) as cloudstorage_file:
                   cloudstorage_file.write('abcde\n')
                   cloudstorage_file.write('f'*1024*4 + '\n')
       self.tmp_filenames_to_clean_up.append(filename) 

       with cloudstorage.open(
           filename, 'w', content_type='text/plain', options={
               'x-goog-meta-foo': 'foo', 'x-goog-meta-bar': 'bar'},
               retry_params=write_retry_params) as cloudstorage_file:
                   cloudstorage_file.write('abcde\n')
                   cloudstorage_file.write('f'*1024*4 + '\n')

Where filename is /bucket_name/file_suffix

4. After calling create_file via a route in your WSGI app, your file will be available at:

    localhost:port/_ah/gcs/bucket_name/file_suffix

Where port is by default 8080, and the file was written to: /bucket_name/file_suffix

Postscript

Unfortunately, I did not find either 3) or 4) in their docs, so I hope this helps someone get set up more easily in the future.

Bill Zito
  • 61
  • 1
  • 3
2

To access gcs objects on dev_appserver, you must specify the bucket & object name, i.e. /_ah/gcs/[bucket]/[object].

Mars
  • 1,422
  • 8
  • 9
0

The storage simulator for the local server is working in later versions of the SDK. For Java, one may choose to follow a dedicated tutorial: “App Engine and Google Cloud Storage Sample”.

George
  • 1,488
  • 1
  • 10
  • 13