1

I am working on a news app and would like to cache the news images on my own google cloud storage.

I am planning to use Flask on GAE. All examples I have found relate to upload a file from user's browser into cloud storage.

What is the best way to obtain an image via an url and upload it to google cloud storage? I hope this makes sense, please feel free to suggest improvements. Many Thanks

def main():
    bucket_name = os.environ.get('BUCKET_NAME',
                                 app_identity.get_default_gcs_bucket_name())
    bucket = '/' + bucket_name
    filename = bucket + '/image_name'
    image_url = "http://news.com/crash.jpg"
    try:
      create_file(image_url, filename)
    except Exception, e:
        logging.exception(e)
    return "Success", 201


def create_file(image_url, filename):
    image = cStringIO.StringIO(urllib.urlopen(image_url).read())  // Not sure about this
    img = Image.open(image)
    write_retry_params = gcs.RetryParams(backoff_factor=1.1)
    gcs_file = gcs.open(filename,
                    'w',
                    content_type='image/jpeg',  // ???? Is MIME type correct?
                    options={'x-goog-acl': 'public'},
                    retry_params=write_retry_params)
    gcs_file.write(img)
    gcs_file.close()
Houman
  • 64,245
  • 87
  • 278
  • 460

1 Answers1

4

Try this:

import urllib2
from google.appengine.api import images
import cloudstorage as gcs

image_at_url = urllib2.urlopen(url) 
content_type =  image_at_url.headers['Content-Type']
filename = #use your own or get from file

image_bytes = image_at_url.read()
image_at_url.close()
image = images.Image(image_bytes)

# this comes in handy if you want to resize images:
if image.width > 800 or image.height > 800:
    image_bytes = images.resize(image_bytes, 800, 800)

options={'x-goog-acl': 'public-read', 'Cache-Control': 'private, max-age=0, no-transform'}
with gcs.open(filename, 'w', content_type=content_type, options=options) as f:
    f.write(image_bytes)
    f.close()
Dmytro Sadovnychyi
  • 6,171
  • 5
  • 33
  • 60
GAEfan
  • 11,244
  • 2
  • 17
  • 33
  • Many thanks. 1) I don't think you need `image = images.Image(image_bytes)` anymore. 2) the code looks very promising, but it doesn't work. Have you tried it? – Houman Dec 23 '14 at 16:53
  • Yes, it does work. i use this with my Flask apps. Make sure you give the GAE app permission to write to your bucket: https://developers.google.com/appengine/docs/python/googlestorage/index#Prerequisites I added a comment about why I use the Image transformation – GAEfan Dec 23 '14 at 17:01
  • It works now. Many thanks for your help. Great idea with resizing images. I suppose having a requirement like this means, I can't use BlobStorage and have to stick to Cloud Storage, since the former's file write is deprecated. Thanks – Houman Dec 23 '14 at 18:12