Batch loading an array with javascript

Question

Possible Duplicate:
Wait until all jquery ajax request are done?

I have an array with an N size. Each element of the array needs to loaded via ajax using jquery. I have the loading logic complete, I am just trying now to figure out how to load only 10 (formula should be able to handle changing this value) at a time, when the 10 items complete loading via ajax, load the next 10. Here is my example.

I have 100 elements of an array, the first 0-9 items need to be loaded, when those 10 items are done, 10-19, then 20-29 and etc. I am trying to get this to be efficient as possible, thanks for any help.

Though this may be totally off, I am hoping I am getting my point across to receive any help.

//Formula calculation
while(wait till count is complete){

}
function LoadBatch(array){
$.each(array,function(i,item){
$.ajax({
success:function(){
//Maybe a counter here to let loop know when to kick off next batch.
}
});
});
}

http://stackoverflow.com/questions/3709597/wait-until-all-jquery-ajax-request-are-done — Andreas, Oct 17 '12 at 06:24

hurrymaplelad · Accepted Answer · 2012-10-17T07:37:07.460

Using a control flow library will make your life easier. Aysnc.queue() looks appropriate. It'll make sure no more than 10 requests are active at once. It won't wait for the previous 10 to finish before starting the next load. This should minimize load time while limiting concurrent requests.

Here's an example:

var results = [];
var concurrency = 10;

var q = async.queue(function (task, done) {
  var item = task.item;
  $.ajax({
    success: function(result) {
      // Store results in array with indicies matching the original array.
      // If you don't care about the order of the results you could just 
      // push them on.
      results[task.index] = result;
      done();
  });
}, concurrency);

// push all the array items into the queue, keeping track of the index
$.each(array, function(i, item) {
  q.push({
    index: i,
    item: item
  });
});

// drain will be called when all the requests are done
q.drain = function() {
  console.log(results); // results has all the responses
}

Thanks. Great start to the answer I was looking for. Cool plugin btw. — user516883, Oct 17 '12 at 16:14

score 0 · Answer 2 · answered Oct 17 '12 at 07:22

Do something as follows:

function loadArray(array, batchSize, callback) {
    var length = array.length
    batchSize = length < batchSize ? length : batchSize; // minimum
    var batch = array.slice(0, batchSize); // the current batch
    array = array.slice(batchSize); // the rest of the array

    loadBatch(batch, function (results) {
        if (array.length) { // there are more batches to process
            // load the rest of the array
            loadArray(array, batchSize, function (nextResults) {
                // merge the results - recursion handles the rest
                callback(results.concat(nextResults));
            });
        } else callback(results); // last batch
    });
}

The loadBatch function is as follows:

function loadBatch(batch, callback) {
    var completed = 0;
    var length = batch.length;
    var results = new Array(length); // the array of results

    $.each(array, function (index, element) {
        $.ajax(element, {
            complete: function (xhr, status) {
                // save the results
                results[index] = {
                    xhr: xhr,
                    status: status
                };

                if (++completed === length) callback(results); // done
            }
        });
    });
}

Now you can load your resources as follows:

loadArray(["a.json", "b.txt", ...], 10, function (results) {
    var a = JSON.parse(results[0]);
    var b = results[1];
    // and so on
});

That's it. Tell me if you have any problems.

Batch loading an array with javascript

2 Answers2