How to pass a blob from a Chrome extension to a Chrome app

Question

A Little Background

I've been working for a couple of days on a Chrome extension that takes a screenshot of given web pages multiple times a day. I used this as a guide and things work as expected.

There's one minor requirement extensions can't meet, though. The user must have access to the folder where the images (screenshots) are saved but Chrome Extensions don't have access to the file system. Chrome Apps, on the other hand, do. Thus, after much looking around, I've concluded that I must create both a Chrome Extension and a Chrome App. The idea is that the extension would create a blob of the screenshot and then send that blob to the app which would then save it as an image to a user-specified location. And that's exactly what I'm doing — I'm creating a blob of the screentshot on the extension side and then sending it over to the app where the user is asked to choose where to save the image.

The Problem

Up to the saving part, everything works as expected. The blob is created on the extension, sent over to the app, received by the app, the user is asked where to save, and the image is saved.... THAT is where things fall apart. The resulting image is unusable. When I try to open it, I get a message that says "Can't determine type". Below is the code I'm using:

1. First ON THE EXTENSION side, I create a blob and send it over, like this:

    chrome.runtime.sendMessage(
       APP_ID, /* I got this from the app */
       {myMessage: blob}, /* Blob created previously; it's correct */
       function(response) {
         appendLog("response: "+JSON.stringify(response));
       }
    );
   

2. Then, ON THE APP side, I receive the blob and attempt to save it like this:

   // listen for external messages
   chrome.runtime.onMessageExternal.addListener(
     function(request, sender, sendResponse) {
       if (sender.id in blacklistedIds) {
         sendResponse({"result":"sorry, could not process your message"});
         return;  // don't allow this extension access
       } else if (request.incomingBlob) {
         appendLog("from "+sender.id+": " + request.incomingBlob);
   
         // attempt to save blob to choosen location
         if (_folderEntry == null) {
            // get a directory to save in if not yet chosen
            openDirectory();
         }
         saveBlobToFile(request.incomingBlob, "screenshot.png");
   
         /*
         // inspect object to try to see what's wrong
         var keys = Object.keys(request.incomingBlob);
         var keyString = "";
         for (var key in keys) {
            keyString += " " + key;
         }
         appendLog("Blob object keys:" + keyString);
         */
   
         sendResponse({"result":"Ok, got your message"});
       } else {
         sendResponse({"result":"Ops, I don't understand this message"});
       }
     }
   );
   

   Here's the function ON THE APP that performs the actual save:

   function saveBlobToFile(blob, fileName) {
     appendLog('entering saveBlobToFile function...');
     chrome.fileSystem.getWritableEntry(_folderEntry, function(entry) {         
       entry.getFile(fileName, {create: true}, function(entry) {         
         entry.createWriter(function(writer) {
           //writer.onwrite = function() {
           //   writer.onwrite = null;
           //   writer.truncate(writer.position);
           //};
           appendLog('calling writer.write...');
           writer.write(blob);                       
           // Also tried writer.write(new Blob([blob], {type: 'image/png'}));
         });
       });
     });
   }
   

There are no errors. No hiccups. The code works but the image is useless. What exactly am I missing? Where am I going wrong? Is it that we can only pass strings between extensions/apps? Is the blob getting corrupted on the way? Does my app not have access to the blob because it was created on the extension? Can anyone please shed some light?

UPDATE (9/23/14) Sorry for the late update, but I was assigned to a different project and could not get back to this until 2 days ago.

So after much looking around, I've decided to go with @Danniel Herr's suggestion which suggests to use a SharedWorker and a page embedded in a frame in the app. The idea is that the Extension would supply the blob to the SharedWorker, which forwards the blob to a page in the extension that is embedded in a frame in the app. That page, then forwards the blob to the app using parent.postMessage(...). It's a bit cumbersome but it seems it's the only option I have.

Let me post some code so that it makes a bit more sense:

Extension:

var worker = new SharedWorker(chrome.runtime.getURL('shared-worker.js'));
worker.port.start();
worker.postMessage('hello from extension'); // Can send blob here too
worker.port.addEventListener("message", function(event) {
   $('h1Title').innerHTML = event.data;
});

proxy.js

var worker = new SharedWorker(chrome.runtime.getURL('shared-worker.js'));
worker.port.start();

worker.port.addEventListener("message",
   function(event) {      
      parent.postMessage(event.data, 'chrome-extension://[extension id]');
   }
);

proxy.html

<script src='proxy.js'></script>

shared-worker.js

var ports = [];
var count = 0;
onconnect = function(event) {
    count++;
    var port = event.ports[0];
    ports.push(port);
    port.start(); 

    /* 
    On both the extension and the app, I get count = 1 and ports.length = 1
    I'm running them side by side. This is so maddening!!!
    What am I missing?
    */
    var msg = 'Hi, you are connection #' + count + ". ";
    msg += " There are " + ports.length + " ports open so far."
    port.postMessage(msg);

    port.addEventListener("message",       
      function(event) {
        for (var i = 0; i < ports.length; ++i) {
            //if (ports[i] != port) {
                ports[i].postMessage(event.data);
            //}
        }
    });
};

On the app

context.addEventListener("message", 
    function(event) {
        appendLog("message from proxy: " + event.data);
    } 
);

So this is the execution flow... On the extension I create a shared worker and send a message to it. The shared worker should be capable of receiving a blob but for testing purposes I'm only sending a simple string.

Next, the shared worker receives the message and forwards it to everyone who has connected. The proxy.html/js which is inside a frame in the app has indeed connected at this point and should receive anything forwarded by the shared worker.

Next, proxy.js [should] receives the message from the shared worker and sends it to the app using parent.postMessage(...). The app is listening via a window.addEventListener("message",...).

To test this flow, I first open the app, then I click the extension button. I get no message on the app. I get no errors either.

The extension can communicate back and forth with the shared worker just fine. The app can communicate with the shared worker just fine. However, the message I sent from the extension->proxy->app does not reach the app. What am I missing?

Sorry for the long post guys, but I'm hoping someone will shed some light as this is driving me insane.

Thanks

Answer 1

Thanks for all your help guys. I found the solution to be to convert the blob into a binary string on the extension and then send the string over to the app using chrome's message passing API. On the app, I then did what Francois suggested to convert the binary string back a blob. I had tried this solution before but I had not worked because I was using the following code on the app:

blob = new Blob([blobAsBinString], {type: mimeType});

That code may work for text files or simple strings, but it fails for images (perhaps due to character encoding issues). That's where I was going insane. The solution is to use what Francois provided since the beginning:

var bytes = new Uint8Array(blobAsBinString.length);
for (var i=0; i<bytes.length; i++) {
   bytes[i] = blobAsBinString.charCodeAt(i);            
}             
blob = new Blob([bytes], {type: mimeString});

That code retrains the integrity of the binary string and the blob is recreated properly on the app.

Now I also incorporated something I found suggested by some of you here and RobW elsewhere, which is to split the blob into chunks and send it over like that, in case the blob is too large. The entire solution is below:

ON THE EXTENSION:

function sendBlobToApp() {  

  // read the blob in chunks/chunks and send it to the app
  // Note: I crashed the app using 1 KB chunks. 1 MB chunks work just fine. 
  // I decided to use 256 KB as that seems neither too big nor too small
  var CHUNK_SIZE = 256 * 1024;
  var start = 0;
  var stop = CHUNK_SIZE;      

  var remainder = blob.size % CHUNK_SIZE;
  var chunks = Math.floor(blob.size / CHUNK_SIZE);      

  var chunkIndex = 0;

  if (remainder != 0) chunks = chunks + 1;           

  var fr = new FileReader();
  fr.onload = function() {
      var message = {
          blobAsText: fr.result,
          mimeString: mimeString,                 
          chunks: chunks 
      };          
      // APP_ID was obtained elsewhere
      chrome.runtime.sendMessage(APP_ID, message, function(result) {
          if (chrome.runtime.lastError) {
              // Handle error, e.g. app not installed
              // appendLog is defined elsewhere
              appendLog("could not send message to app");
          } 
      });

      // read the next chunk of bytes
      processChunk();
  };
  fr.onerror = function() { appendLog("An error ocurred while reading file"); };
  processChunk();

  function processChunk() {
     chunkIndex++;         

     // exit if there are no more chunks
     if (chunkIndex > chunks) {
        return;
     }

     if (chunkIndex == chunks && remainder != 0) {
        stop = start + remainder;
     }                           

     var blobChunk = blob.slice(start, stop);

     // prepare for next chunk
     start = stop;
     stop = stop + CHUNK_SIZE;

     // convert chunk as binary string
     fr.readAsBinaryString(blobChunk);
  } 
}

---

ON THE APP

chrome.runtime.onMessageExternal.addListener(
  function(request, sender, sendResponse) {
    if (sender.id in blacklistedIds) {
      return;  // don't allow this extension access
    } else if (request.blobAsText) {                  
       //new chunk received  
      _chunkIndex++;                   

      var bytes = new Uint8Array(request.blobAsText.length);                     
      for (var i=0; i<bytes.length; i++) {
         bytes[i] = request.blobAsText.charCodeAt(i);            
      }         
      // store blob
      _blobs[_chunkIndex-1] = new Blob([bytes], {type: request.mimeString});           

      if (_chunkIndex == request.chunks) {                      
         // merge all blob chunks
         for (j=0; j<_blobs.length; j++) {
            var mergedBlob;
            if (j>0) {                  
               // append blob
               mergedBlob = new Blob([mergedBlob, _blobs[j]], {type: request.mimeString});
            }
            else {                  
               mergedBlob = new Blob([_blobs[j]], {type: request.mimeString});
            }
         }                         

         saveBlobToFile(mergedBlob, "myImage.png", request.mimeString);
      }
    }
 }
);

Answer 2

Does my app not have access to the blob because it was created on the extension? Can anyone please shed some light?

Exactly! You may want to pass a dataUrl instead of a blob. Something like this below could work:

/* Chrome Extension */

var blobToDataURL = function(blob, cb) {
  var reader = new FileReader();
  reader.onload = function() {
    var dataUrl = reader.result;
    var base64 = dataUrl.split(',')[1];
    cb(base64);
  };
  reader.readAsDataURL(blob);
  };

blobToDataUrl(blob, function(dataUrl) {
  chrome.runtime.sendMessage(APP_ID, {databUrl: dataUrl}, function() {});
});

/* Chrome App */

function dataURLtoBlob(dataURL) {
    var byteString = atob(dataURL.split(',')[1]),
        mimeString = dataURL.split(',')[0].split(':')[1].split(';')[0];

    var ab = new ArrayBuffer(byteString.length);
    var ia = new Uint8Array(ab);
    for (var i = 0; i < byteString.length; i++) {
        ia[i] = byteString.charCodeAt(i);
    }

    var blob = new Blob([ia], {type: mimeString});
    return blob;
}

chrome.runtime.onMessageExternal.addListener(
    function(request) {  
  var blob =  dataURLtoBlob(request.dataUrl); 
  saveBlobToFile(blob, "screenshot.png");
});

Answer 3

I am extremely interested in this question, as I am trying to accomplish something similar.

these are the questions that I have found to be related:

1. How can a Chrome extension save many files to a user-specified directory?
2. Implement cross extension message passing in chrome extension and app
3. Does chrome.runtime support posting messages with transferable objects?
4. Pass File object to background.js from content script or pass createObjectURL (and keep alive after refresh)

According to Rob W, in the first link:

"Chrome's fileSystem (app) API can directly write to the user's filesystem (e.g. ~/Documents or %USERPROFILE%\Documents), specified by the user."

If you can write to a user's filesystem you should be able to read from it right?

I haven't had the opportunity to try this out, but instead of directly passing the file blob to the app, you could save the item to your downloads using the chrome extension downloads api.

Then you could retrieve it with the chrome app filesystem api to gain access to it.

Edit:

I keep reading that the filesystem the api can access is sandboxed. So I have no idea if this solution is possible. It being sandboxed and Rob W's description of "writing directly to the user's filesystem" sound like opposites to me.

Edit:

Rob W has revised his answer here: Implement cross extension message passing in chrome extension and app.

It no longer uses a shared worker, and passes file data as a string to the backend, which can turn the string back into a blob.

I'm not sure what the max length of a message is, but Rob W also mentions a solution for slicing up blobs to send them in pieces.

Edit:

I have sent 43 mbs of data without crashing my app.

Answer 4

That's really an intresting question. From my point of view it can be done using these techniques:

1. First of all you should convert your blob to arraybuffer. This can be done with FileReader, and it is async operation
2. Then here comes some magic of Encoding API, which is currently available on stable Chrome. So you convert your arraybuffer into string. This operation is sync
3. Then you can communicate with other extensions/apps using Chrome API like this. I am using this technique to promote one of my apps (new packaged app) using another famous legacy app. And due to the fact that legacy packaged apps are in fact extensions, I think everything will be okay.