How to wait for a large, arbitrary number of deeply nested async jQuery ajax calls to complete

Question

I am using the SharePoint rest web service interface to get a large amount of data from the site.

This involves deeply nested ajax calls to different services (i.e. get website, get it's lists, get all the list's items, for each item, get it's properties, get the webhsite's sub site's... repeat).

I am using an array of promises, pushing each call into the array:

 promises.push($.ajax({
        url: fullurl,
        method: "GET",
        headers: { "Accept": "application/json; odata=verbose" },
        success: function (data) {                  
                  $.each(data.d.results, function(i, item) {
                ProcessData(i,item, tableTarget, columns, promises);                                                
                   });      
        },
        error: function (data) {
                console.log("request failed: " + fullurl);
        }
    }));

Each call to ProcessData potentially adds a line to an html table ('tableTarget') and calls for more data.

Ideally I want to delay scripts that attempt to filter the table until all the data is there. The problem is that when I make a call to

var promises = [];
//kickoff my data gathering
AddWebData(url,tableTarget, promises);
//apply
$.when.apply($, promises).then(processFinalTable(promises));

There are only about 2 items in the promises array, so the processFinalTable is called too early. If I delay

setTimeout(function(){ console.log('waited 4000'); $.when.apply($, promises).then(processFinalTable(promises));
}, 4000);

The promises list has 400 or so items. In total I actually need to wait for 1200 odd ajax calls (don't ask).

Given that I am going to make an arbitrary number of calls, depending on the SharePoint site data. How do I determine when the last of my async calls has completed?

you can have each callback clearTimeout() and setTimeout(done, 500); , where done is a function that sorts. that way, URLs will usually staircase in and prevent the sort by re-delaying it. if it stalls for a bit, the sort will be called, as it will after the connections pick back up. if the sort is not-destructive or happen "under the fold", you don't even need to redraw each time, and worst-case, you're only drawing twice a second with the 500ms delay. the 500ms mentioned should actually ne be no more than about twice the average turnaround time for these ajax requests. — dandavis, Aug 12 '14 at 03:34

klenwell · Answer 1 · 2014-08-13T03:52:11.673

How about something like this? Encapsulate your calls in a function that returns a deferred object that gets resolved once all your calls are complete. Code is commented to help clarify what is going on:

MIN_DELAY = 200;
MAX_DELAY = 2000;

function mockAjaxCall() {
  var deferred = jQuery.Deferred();
  var ms_delay = Math.floor(Math.random() * (MAX_DELAY - MIN_DELAY + 1)) + MIN_DELAY;

  setTimeout(function() {
    deferred.resolve(ms_delay);
  }, ms_delay);

  return deferred;
}

function collectAllThatData() {
  var allCallsComplete = jQuery.Deferred();
  var callsPending = [];
  var callsComplete = [];
  var ms_interval = 250;

  // Let's mock 40 asynchronous AJAX calls with random delays
  for (var i=0; i<40; i++) {
    var promise = mockAjaxCall();
    callsPending.push(promise);
  }

  // As promises are resolved, add to callsComplete array.
  jQuery.each(callsPending, function(n, callPending) {
    jQuery.when(callPending).done(function() {
      callsComplete.push(callPending);
    });
  });

  // At defined interval, compare callsComplete to callsPending to determine
  // if all our calls are complete.
  var intervalId = setInterval(function() {
    console.log(callsComplete.length, 'calls complete');

    if ( callsComplete.length >= callsPending.length ) {
      clearInterval(intervalId);
      allCallsComplete.resolve(callsComplete);
    }
  }, ms_interval);

  // Return a deferred object for calling script to use.
  return allCallsComplete;
}

var allThatDataCollected = collectAllThatData();

jQuery.when(allThatDataCollected).then(function() {
  console.log('Data has been collected!');
  // Do something else
});

Fiddle to demonstrate:

http://jsfiddle.net/klenwell/jyza3050/

score 1 · Accepted Answer · answered Aug 14 '14 at 20:25

1

Unfortunately the answer is that I used incorrect syntax.

I changed:

$.when.apply($, promises).then(processFinalTable());

to:

$.when.apply($, promises).then(processFinalTable);

And now everything works correctly.

answered Aug 14 '14 at 20:25

Nat

14,175
5
41
64

Please do you think you help me with this question : http://stackoverflow.com/questions/25510493/how-to-test-webservice-simulating-serveral-concurrent-requests – Axel Aug 26 '14 at 16:29

How to wait for a large, arbitrary number of deeply nested async jQuery ajax calls to complete

2 Answers2