2

I'd like to retrieve and parse multiple html pages within a Chrome extension.

Using Web Workers for each request seemed like a simple way to make them execute in parallel. Is that possible? My attempt failed, perhaps because it's a known permissions bug.

As a workaround, I guess I could have the main extension page do multiple asynchronous XmlHttpRequests, then have the callback send the result page to Web Workers for parallel parsing. But that method raises the question of how many asynchronous parallel requests can Chrome make at once? That question has been asked here, without answer.

The Chrome Extension I'm writing is a Browser Action.

Code for the worker:


// Triggered by postMessage in the page
onmessage = function (evt) {
    var message = evt.data;
    postMessage(message.count + " started");
    update(message.count, message.url);
};

// parse the results page
function parseResponse(count, url, resp) {
    var msg = count.toString() + " received response ";
    postMessage(msg);
}

// read the Buganizer URL and parse the result page
var update = function(count, url) {
  var xhr = new XMLHttpRequest();
      xhr.onreadystatechange = function() {
        if (xhr.readyState == 4 && xhr.status == 200) {
          parseResponse(count, url, xhr.responseText);
        }
      }
  xhr.onerror = function(error) {
      var msg = "!>: " + count + ": error ";
      postMessage(msg);
  }
  var url = "http://www.hotgrog.com"; // for testing (manifest has permissions for this url)
  xhr.open("GET", url, true);
  xhr.send();
  postMessage(url);
}
Community
  • 1
  • 1
DrGary
  • 3,321
  • 3
  • 20
  • 14
  • Are you positive that parallel parsing is needed? I have a feeling that 99% of the time will be spent on requesting pages and 1% on parsing. In this case workers aren't really useful. Ajax requests are already async so whether you put them into workers or not it doesn't change anything - you will hit exactly the same network limitations. – serg May 11 '11 at 18:33
  • Y, so if I don't use separate workers, I'm left with the question of how many XmlHttpRequests can occur concurrently in Chrome? – DrGary May 11 '11 at 21:26
  • I guess, which is answered in that question you linked (second answer from the top). – serg May 11 '11 at 21:33

1 Answers1

0

Have you looked into trying asynchronous-loaders such as RequireJS or Curl?

Take a look at the authors explanation as to WHY we should use his product.

Prisoner ZERO
  • 13,848
  • 21
  • 92
  • 137