using appengine blobs for binary data in an obj-c app

Question

I'm writing an obj-c app and would like to upload a binary file a few megs in size to my appengine server (python). I'm guessing I need to use the blob entity for this, but am unsure how to go about doing this. I've been using http requests and responses to send and receive data up to now, but they've been encoded in strings. Can someone advise how I'd go about doing the same with blobs from an obj-c app? I see some examples that involve http requests but they seem geared toward web page and I'm not terribly familiar with it. Are there any decent tutorials or walkthroughs perhaps?

I'm basically not completely sure, if I'm supposed to encode it into the http request and send it back through the response, how to get the binary data into the http string from the client and how to send it back properly from the server when downloading my binary data. I'm thinking perhaps the approach has to be totally different from what I'm used to with encoding values into my request in the param1=val&param2=val2 style format but uncertain.

Should I be using the blobstore service for this? One important note is that I've heard there is a 1 meg limit on blobs, but I have audio files 2-3 megs in size that I need to store (at the very least 1.8 megs).

Answer 1

I recently had to do something similar, though it was binary data over a socket connection. To the client using XML, to the server as a data stream. I ended up base64 encoding the binary data when sending it back and forth. It's a bit wordy but especially on the client side it made things easier to deal with, no special characters to worry about in my XML. I then translated it with NSData into a real binary format. I used this code to do the encoding and decoding, search for "cyrus" to find the snippet I used, there are a few that would work here.

In your case I would change your http request to a post data call rather than putting it all in the URL. If you're not sure what the difference is, have a look here.

I'm not as familiar with python, but you could try here for help on that end.

Hope that helps.

Edit - it looks like blobs are the way to go. Have a look at this link for the string/blob type as well as this link for more info on working with the blob.

Answer 2

There are three questions in one here:

1. Should you use a BLOB for binary data?
2. How do you post binary data, and use it from app engine
3. How do you retrieve binary data from app engine

I can't answer if you "should" use blobs, only you would know the answer to that, and it greatly depends upon the type of data you are trying to store, and how it will be used. Let's take an image for example (which is probably the most popular use case for this). You want users to take a photo with their phone, upload it, and then share it with other users. That's a good use of blobs, but as @slycrel suggests you'll run into limitations on record size. This can be workable, for example you could use the python image library (pil) to downsize the image.

To post binary data, see this question. It would be best to cache 2 copies, a thumbnail and a full size. This way the resizing only has to happen once, on upload. If you want to go one better, you can use the new background jobs feature of app engine to queue up the image processing for later. Either way, you'll want to return the ID of the newly created blob so you can reference it from the device without an additional http request.

To retrieve data, I think the best approach would be to treat the BLOB as it's own resource. Adjust your routes such that any given blob has a unique URL:

http://myweb/images/(thumbnail|fullsize)/<blobid>.(jpg|png|gif)

Where BLOBID is dynamic, and JPG, PNG or GIF could be used to get the particular type of image. Thumbnail or fullsize could be used to retrieve the smaller or larger version you saved when they posted it.