Concept - Designing a collapsible queue for asynchronous resources

Question

I've noticed that the size of a file requested will effect how long the response takes for ajax calls. So if I fire 3 ajax GET requests for files of varying size, they may arrive in any order. What I want to do is guarantee the ordering when I append the files to the DOM.

How can I set up a queue system so that when I fire A1->A2->A3. I can guarantee that they are appeneded as A1->A2->A3 in that order.

For example, suppose A2 arrives before A1. I would want the action to wait upon the arrival and loading of A1.

One idea is to create a status checker using a timed callback as such

// pseudo-code
function check(ready, fund) {
    // check ready some how
    if (ready) {
        func();
    } else {
        setTimeout(function () {
            check(ready, fund);
        }, 1); // check every msec
    }
}

but this seems like a resource heavy way, as I fire the same function every 1msec, until the resources is loaded.

Is this the right path to complete this problem?

Answer 1

status checker using a 1msec-timed callback - but this seems like a resource heavy way; Is this the right path to complete this problem?

No. You should have a look at Promises. That way, you can easily formulate it like this:

var a1 = getPromiseForAjaxResult(ressource1url);
var a2 = getPromiseForAjaxResult(ressource2url);
var a3 = getPromiseForAjaxResult(ressource3url);

a1.then(function(res) {
    append(res);
    return a2;
}).then(function(res) {
    append(res);
    return a3;
}).then(append);

For example, jQuery's .ajax function implements this.

Answer 2

You can try something like this:

var resourceData = {};
var resourcesLoaded = 0;

function loadResource(resource, callback) {
    var xhr = new XMLHttpRequest();
    xhr.onload = function() {
        var state = this.readyState;
        var responseCode = request.status;

        if(state == this.DONE && responseCode == 200) {
            callback(resource, this.responseText);
        }
    };

    xhr.open("get", resource, true);
    xhr.send();
}

//Assuming that resources is an array of path names
function loadResources(resources) {
    for(var i = 0; i < resources.length; i++) {
        loadResource(resources[i], function(resource, responseText) {

            //Store the data of the resource in to the resourceData map,
            //using the resource name as the key. Then increment the
            //resource counter.
            resourceData[resource] = responseText;
            resourcesLoaded++;

            //If the number of resources that we have loaded is equal
            //to the total number of resources, it means that we have
            //all our resources.
            if(resourcesLoaded === resources.length) {
                //Manipulate the data in the order that you desire.
                //Everything you need is inside resourceData, keyed
                //by the resource url. 
                ...
                ...
            }                    
        });
    }
}

If certain components must be loaded and executed before (like certain JS files) others, you can queue up your AJAX requests like so:

function loadResource(resource, callback) {
    var xhr = new XMLHttpRequest();
    xhr.onload = function() {
        var state = this.readyState;
        var responseCode = request.status;

        if(state == this.DONE && responseCode == 200) {
            //Do whatever you need to do with this.responseText
            ...
            ...

            callback();
        }
    };

    xhr.open("get", resource, true);
    xhr.send();
}

function run() {
    var resources = [
        "path/to/some/resource.html",
        "path/to/some/other/resource.html",
        ...
        "http://example.org/path/to/remote/resource.html"
    ];

    //Function that sequentially loads the resources, so that the next resource 
    //will not be loaded until first one has finished loading. I accomplish
    //this by calling the function itself in the callback to the loadResource 
    //function. This function is not truly recursive since the callback 
    //invocation (even though it is the function itself) is an independent call 
    //and therefore will not be part of the original callstack.
    function load(i) {
        if (i < resources.length) {
            loadResource(resources[i], function () {
                load(++i);
            });
        }
    }
    load(0);
}

This way, the next file will not be loaded until the previous one has finished loading.

If you cannot use any third-party libraries, you can use my solution. However, your life will probably be much easier if you do what Bergi suggested and use Promises.

Answer 3

There's no need to call check() every millisecond, just run it in the xhr's onreadystatechange. If you provide a bit more of your code, I can explain further.

Answer 4

I would have a queue of functions to execute and each of them checks the previous result has completed before executing.

var remoteResults[] 

function requestRemoteResouse(index, fetchFunction) {
  // the argument fetchFunction is a function that fetches the remote content
  // once the content is ready it call the passed in function with the result.
  fetchFunction(
    function(result) { 
      // add the remote result to the list of results
      remoteResults[index] = result
      // write as many results as ready.
      writeResultsWhenReady(index);
    });
}

function writeResults(index) {
  var i;
  // Execute all functions at least once
  for(i = 0; i < remoteResults.length; i++) {
    if(!remoteResults[i]) {
      return;
    }
    // Call the function that is the ith result
    // This will modify the dom.
    remoteResults[i]();
    // Blank the result to ensure we don't double execute
    // Store a function so we can do a simple boolean check.
    remoteResults[i] = function(){}; 
  }
}

requestRemoteResouse(0, [Function to fetch the first resouse]);
requestRemoteResouse(1, [Function to fetch the second resouse]);
requestRemoteResouse(2, [Function to fetch the thrid resouse]);

Please note that this is currently O(n^2) for simplicity, it would get faster but more complex if you stored an object at every index of remoteResults, which had a hasRendered property. Then you would only scan back until you found a result that had not yet occurred or one that has been rendered.