Clean way to make Subsequent AJAX Calls to API based on Data

Question

So I have a conceptual question regarding the cleanest way to make subsequent AJAX calls to an API based on the returned data.

A quick example:

A function, which encompasses the call would look like this:

function makeCall(headers, min, max) {
    $.ajax({ 
        headers: headers, 
        url: "https://coolapi.com/data?begIndex" + min + "&endIndex=" + max + "&begTimestamp=1404198000000&endTimestamp=1409554800000", 
        type: "GET", 
        dataType: 'JSON'
    });
}

makeCall(headers, 0, 20);

The beg / end index (min/max), determine the amount of data I'll get back in the array. The API will only return a maximum of 20 items in the array, but it will also return me a COUNT of how many items total exist in that array. An example of the data returned is below:

{
    count = 133;
    result = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19];   
}

So my next call would be:

makeCall(headers, 20, 40);

and so on so forth, until I got all 133 items from the array.

The question is...what is the cleanest way to continue to make subsequent calls until I've gotten and stored all 133 items from the array? Given that the count could be any number, it's hard to imagine how I can do this. I was thinking of nesting more ajax calls in a "success" function, but it's not scalable if I get back a number like 300.

Does anyone have any advice on how to proceed?

Thanks in advance!

EDIT:

So based on the advice in the comment, I've attemped to make the call recursive--but it doesn't seem to function as intended:

var theData = [];
var minCounter=0;
var maxCounter= minCounter + 20;

function makeCall(headers, min, max) {
$.ajax({ 
    headers: headers, 
    url: "https://coolapi.com/data?begIndex" + min + "&endIndex=" + max + "&begTimestamp=1404198000000&endTimestamp=1409554800000", 
    type: "GET", 
    dataType: 'JSON',
    success: function (data) {
        theData.push(data.result);
        newMin = minCounter + 20;
        if (data.count >= theData.length ) {
            makeCall(headers, newMin, maxCounter);
        }
    }
});
}

makeCall(headers, minCounter, maxCounter);

How do properly increment the variable as well as set the flag?

SECOND EDIT:

The method below works using the second comment's suggestion, but there are some issues here as well...

function doAjax(headers, min, dObject) {
    var max = min + 20;
    $.ajax({
        headers: headers, 
        url: "https://coolapi.com/data?begIndex" + min + "&endIndex=" + max + "&begTimestamp=1404198000000&endTimestamp=1409554800000", 
        type: "GET", 
        dataType: 'JSON',
        success: function (data) {
            results.push(data);
            window.count = data.count;
            dObject.resolve();
        }
    });
}

// array that will contain all deferred objects
var deferreds = [];

// array that will contain all results
var results = [];

// make the ajax calls
for (var i = 20; i < 133 ; i+= 20) {
    var dObject = new $.Deferred();
    deferreds.push(dObject);
    doAjax(headers, i, dObject);
}

// check if all ajax calls have finished
$.when.apply($, deferreds).done(function() {
    console.log(results);
});

var dObject = new $.Deferred();
doAjax(headers,0, dObject);

First, the data doesn't push to the array in order. There doesn't seem anyway to fix this. Also strangely enough, in the for loop--I have to set the number for it to actually work. Trying to store it in a variable doesn't seem to work as well...Suggestions here?

You should make `makeCall` recursive, with `makeCall` in the success callback of the ajax call. Your ajax response should then return the `max` parameter (to use as `min` in the makeCall in the success callback) as well as a flag to indicate when no more data is available. — mccannf, Aug 08 '14 at 20:09
I would look into [Promise/Deferred](http://api.jquery.com/category/deferred-object/). — Jared Farrish, Aug 08 '14 at 22:17
http://stackoverflow.com/questions/13912775/jquery-deferred-getting-result-of-chained-ajax-calls — Jared Farrish, Aug 08 '14 at 22:25

score 2 · Accepted Answer · answered Aug 10 '14 at 01:33

Here's a working implementation based around the code you started with. Code is commented to help you understand what is happening:

// Change these constants to suit your purposes.
var API_URL = 'https://coolapi.com/data';
var HEADERS = {};
var API_RESULTS_PER_REQUEST = 20;
var MAX_API_CALLS = 20;

// Count API calls to trigger MAX_API_CALLS safety lock.
var apiCalls = 0;

// Function we'll call to get all our data (see bottom).
function collectApiData(begTimestamp, endTimestamp) {
  var dataReady = jQuery.Deferred();

  var params = {
    'begTimestamp': begTimestamp,
    'endTimestamp': endTimestamp
  };

  var datasetsCollected = requestDatasets(params);

  jQuery.when(datasetsCollected).then(function(data) {
    dataReady.resolve(data);
  });

  return dataReady;
}

// Makes individual AJAX call to API
function callApi(params, headers) {
  var $request = jQuery.ajax({
    url: API_URL,
    headers: headers,
    data: params,
    type: 'GET',
    dataType: 'JSON'
  });

  return $request;
}

// Recursive function that makes API calls until data is collected, there is an
// error, or MAX_API_CALLS limit is hit.
function requestDatasets(params, resultsReady, resultsFetched) {
  resultsReady = ( resultsReady !== undefined ) ? resultsReady : jQuery.Deferred();
  resultsFetched = ( resultsFetched !== undefined ) ? resultsFetched : [];

  // Trigger safety to avoid API abuse
  if ( apiCalls >= MAX_API_CALLS ) {
    console.error('Exceeded max API calls:', MAX_API_CALLS);
    resultsReady.resolve(resultsFetched);
  }

  // Set index data
  params.begIndex = resultsFetched.length;
  params.endIndex = resultsFetched.length + API_RESULTS_PER_REQUEST;

  // Request dataset from API
  var apiRequest = callApi(params, HEADERS);
  apiCalls += 1;

  // Callback once API request has completed and data is ready
  jQuery.when(apiRequest).done(function(data) {
    var apiResultCount = data.count;
    resultsFetched = resultsFetched.concat(data.result);
    console.debug('Fetched', resultsFetched.length, 'of', apiResultCount, 'API results');

    if ( apiResultCount > resultsFetched.length ) {
      console.debug('Making another API call');
      requestDatasets(params, resultsReady, resultsFetched);
    }
    else {
      console.debug('Results all fetched!');
      resultsReady.resolve(resultsFetched);
    }
  });

  jQuery.when(apiRequest).fail(function(data) {
    console.error('API error: returning current results.');
    resultsReady.resolve(resultsFetched);
  });

  return resultsReady;
}

// Run script
var dataReady = collectApiData('1404198000000', '1409554800000');
jQuery.when(dataReady).then(function(data) {
  console.log(data);
});

Here's a working fiddle that mocks the API using httpbin.org:

http://jsfiddle.net/klenwell/mfhLxun2/

Thank you very much for this detailed implementation. I learned quite a lot from it. Not only regarding Deferred(), but also about structuring my code and considering what is configuration vs actual application logic (from how you broke the request down into a separate function). Thanks again for this! — richie, Aug 10 '14 at 21:26
@richie I'm glad to hear it helped. It proved more challenging than I expected going in and ended up being a bit of a learning experience for me, too. — klenwell, Aug 10 '14 at 22:26
Also, it's a good practice to store the numbers in a variable with suitable name and pass the variable in a function. — Ashish Neupane, Jul 21 '22 at 04:46

Clean way to make Subsequent AJAX Calls to API based on Data

1 Answers1