Saving canvas image via JavaScript from Safari (5.0.x) to AppEngine Blobstore

Question

My team and I are unhappy that our awesome animation app doesn't work on Safari because of a combination of Safari and AppEngine limitations. We're hoping one of you can help us find the "magic incantation" to solve it.

This may be an extremely easy problem to solve, but we've hit only brick walls after two days, since this is an uncommon (though very useful) scenario that is pretty much undocumented.

Let me explain the details of the problem.

Our app needs to save canvas data to the blobstore (images the user has drawn for their animation.) Usually, you would do this by posting a web form dynamically via ajax that has a binary field in it with the image data. One way to do this is by using ArrayBuffer and BlobBuilder. This works with Chrome:

dataURItoBlob = function(dataURI, callback) {
  var ab, bb, byteString, i, ia, mimeString, _ref;
  if (!(typeof ArrayBuffer != "undefined" && ArrayBuffer !== null)) {
    return null;
  }
  byteString = atob(dataURI.split(',')[1]);
  mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
  ab = new ArrayBuffer(byteString.length);
  ia = new Uint8Array(ab);
  for (i = 0, _ref = byteString.length; (0 <= _ref ? i < _ref : i > _ref); (0 <= _ref ? i += 1 : i -= 1)) {
    ia[i] = byteString.charCodeAt(i);
  }
  bb = window.BlobBuilder ? new BlobBuilder() : window.WebKitBlobBuilder ? new WebKitBlobBuilder() : window.MozBlobBuilder ? new MozBlobBuilder() : void 0;
  if (bb != null) {
    bb.append(ab);
    return bb.getBlob(mimeString);
  } else {
    return null;
  }
};
postCanvasToBlobstore = function(url, name, canvas) {
        blob = dataURItoBlob(canvas.toDataURL());
        formData = new FormData();
        formData.append("file", blob);
        xhr = new XMLHttpRequest();
        xhr.open("POST", url);
        return xhr.send(formData);
}

Another way to save binary form data is to use xhr.sendAsBinary(). This works for Firefox:

postCanvasToBlobstore = function(url, name, canvas) {
  type='image/png'
  var arr, boundary, data, j, xhr;
  data = canvas.toDataURL(type);
  data = data.replace('data:' + type + ';base64,', '');
  xhr = new XMLHttpRequest();
  xhr.open('POST', url, true);
  boundary = 'imaboundary';
  xhr.setRequestHeader('Content-Type', 'multipart/form-data; boundary=' + boundary);
    arr = ['--' + boundary, 'Content-Disposition: form-data; name="' + name + '"; filename="' + name + '"', 'Content-Type: ' + type, '', atob(data), '--' + boundary + '--'];
  j = arr.join('\r\n');
  return xhr.sendAsBinary(j);
}

Neither of these possibilities seem to exist for Safari (though it's very possible it is, but we're not smart enough to figure it out). One alternative is to just use base64 encoded data, which Safari can do for sure. This is what that would look like:

postCanvasToBlobstore = function(url, name, canvas, type) {
  type='image/png'
  var arr, boundary, data, j, xhr;
  data = canvas.toDataURL(type);
  data = data.replace('data:' + type + ';base64,', '');
  xhr = new XMLHttpRequest();
  xhr.open('POST', url, true);
  boundary = 'imaboundary';
  xhr.setRequestHeader('Content-Type', 'multipart/form-data; boundary=' + boundary);
    arr = ['--' + boundary, 'Content-Disposition: form-data; name="' + name + '"; filename="' + name + '"', 'Content-Transfer-Encoding: base64','Content-Type: ' + type, '', data, '--' + boundary + '--'];
  j = arr.join('\r\n');
  return xhr.send(j);
}

Now this actually works! However, it only works on the development version of the AppEngine tools, because of a known blobstore bug: The moment you deploy your app into production it stops working. Of course, it's possible that tweaking the POSTing code somewhere may resolve the problem the blobstore is having interpreting the data. See http://code.google.com/p/googleappengine/issues/detail?id=4265 for what appears to be the blobstore issue connected with this problem.

You can paste any of the three code samples above into the edit control and can see what happens with each version of the postCanvasToBlobstore function: the first sample will work in Chrome, the second will work in Firefox (and is the version the debug app defaults to), and the third should work from all three (but works for none of them when using this production website, probably because of a blobstore bug.)

Answer 1

This appears to be a bug with how the production environment handles Content-Transfer-Encoding. I've filed it internally, but if you don't want to wait for it to be resolved (and I expect you don't), a workaround is in order.

Fortunately, we recently released the Files API, which makes it possible to write to the blobstore programmatically, and we've also increased the limit on upload request bodies to 32MB. As long as your payloads are under that size, you can write a handler to accept the upload in whatever format is most convenient to you, and store it to the blobstore yourself. I would recommend keeping the regular upload mechanism around, since it's more efficient when it's practical to use it, but that's up to you.

Bear in mind that individual calls to App Engine APIs still have a size limit; to write the whole file you'll need to make multiple calls to write. For more details, see my answer to this related question.

Answer 2

What I do is Shard/Chunk the base64 from canvas through rpc about 25000 char through rpc, then I break that down to 900 chars and store them into a tmp table in the datastore as TEXT. Once thats done I do a few checks to make sure its all there. Then I stream decode it into the blobstore.

Here is some of the source: GAEDatastore

Got any more questions: branflake2267

Please star this issue.

Please comment in the App Engine forum so the Engineers see it too:

Hope that helps.