7

I want to manipulate a pickled python object stored in S3 in Google App Engine's sandbox. I use the suggestion in boto's documentation:

from boto.s3.connection import S3Connection

from boto.s3.key import Key

conn = S3Connection(config.key, config.secret_key)
bucket = conn.get_bucket('bucketname')
key = bucket.get_key("picture.jpg")
fp = open ("picture.jpg", "w")
key.get_file (fp)

but this requires me to write to a file, which apparently is not kosher in the GAE sandbox.

How can I get around this? Thanks much for any help

Jonik
  • 80,077
  • 70
  • 264
  • 372
rd108
  • 631
  • 2
  • 7
  • 14

2 Answers2

7

You don't need to write to a file or a StringIO at all. You can call key.get_contents_as_string() to return the key's contents as a string. The docs for key are here.

Nick Johnson
  • 100,655
  • 16
  • 128
  • 198
  • thanks Nick. This works, and without having to import the StringIO module. I think for obvious reasons that makes it a better solution. For anyone following along at home, I changed the pickle.load(content) to pickle.loads(content) to work with unpickling a string-like, rather than file-like, object. – rd108 Oct 19 '10 at 00:45
  • I'd also suggest using validate=False to the get_bucket call - ie bucket = conn.get_bucket(bucket_name, validate=False) as boto attempts to access the bucket, and will fail if it doesn't have access to do so. for more information see: http://stackoverflow.com/questions/12571217/python-amazon-s3-cannot-get-the-bucket-says-403-forbidden – Hamish Currie Aug 09 '13 at 05:20
  • Also, boto keys have a .open() call that you can use. – meawoppl Mar 10 '15 at 18:54
3

You can write into a blob and use the StringIO to retrieve the data

from boto.s3.connection import S3Connection
from boto.s3.key import Key
from google.appengine.ext import db

class Data(db.Model)
    image = db.BlobProperty(default=None)

conn = S3Connection(config.key, config.secret_key)
bucket = conn.get_bucket('bucketname')
key = bucket.get_key("picture.jpg")
fp = StringIO.StringIO()
key.get_file(fp)

data = Data(key_name="picture.jpg")
data.image = db.Blob(fp.getvalue())
data.put()
Shay Erlichmen
  • 31,691
  • 7
  • 68
  • 87