how do I send procedural requests (one after the other) with loop from angular factory?

Question

I am looking to get all artists that a user follows from Spotify. The catch is that I can only request 50 artists at a time and only know the total number of artists after the first request. My factory looks like this:

app.factory('artists', [
    "$rootScope", 
    "$http",
    "$cookies",
    "$q",
    function($rootScope, $http, $cookies, $q){

    return {
        getUserFollowing: function(){
            var allArtists = [];
            var _params = {
                type: 'artist',
                limit: 50, // maximum
            };
            var _authConfig = {
                'Authorization': 'Bearer ' + $cookies.get('spotify-token'),
                'Content-Type': 'application/json'
            };

            var req = {
                method: 'GET',
                url: 'https://api.spotify.com/v1/me/following',
                headers: _authConfig,
                params: _params
            };
            return $http(req).then(function(res){

                var artistTotal = res.data.artists.total;
                allArtists = allArtists.concat(res.data.artists.items);

                if(allArtists.length < artistTotal){
                    // send the request again to load 50 more artists
                    return $http({
                        method: 'GET',
                        url: res.data.artists.next,
                        headers: _authConfig,
                    }).then(function(nextRes){
                        allArtists = allArtists.concat(nextRes.data.artists.items);
                        return allArtists;
                    });
                } else {
                    // need to return a promise
                    var dfd = $q.defer();
                    dfd.resolve(allArtists);
                    return dfd.promise;
                }
            });
        }
    };
}]);

In the above request I send an additional request (for another 50) if the user follows more than 50 artists. If they follow less than 50 artists it will return the total. I am using this factory from a controller like so:

// grab the artists they follow
artists.getUserFollowing().then(function(artistsUserFollows){
    $rootScope.artistTotal = artistsUserFollows.length;
    $rootScope.artists = artistsUserFollows;
});

How can I write a loop so that the getUserFollowing() will return an array of all artists whether they follow no artists or 1000 artists? If they follow 1000 artists then we'd need to send 20 requests (1000/50) or if they follow no artists only one request.

How can I send a dynamic number of requests to get all artists a user follows, 50 at a time? Also the next url for the request (without auth header) is defined in res.data.artists.next

Can you paste a sample of the next URL as returned by spotify? If the format is simple enough, maybe you can construct URLs dynamically once you get the total number of artists and fire multiple requests in parallel. — Digitrance, Nov 16 '15 at 19:55
Next url that spotify returns: https://api.spotify.com/v1/me/following?type=artist&after=58UpHBCQ1Jj67DJsR7Qyqg&limit=50. That's the endpoint for the next request (minus the auth header). Multiple requests not really possible because the `after` param is the final spotify id from the previous result of artists — Connor Leech, Nov 16 '15 at 20:09
@Conner, I reached the same conclusion after going through spotify's API that a parallel solution is not possible. — Digitrance, Nov 16 '15 at 20:14

Digitrance · Accepted Answer · 2015-11-16T21:06:32.623

UPDATED SOLUTION

I think you need a recursive solution for this problem. I have not tested this, but I think it should work. Let me know how it goes, and if you have any questions.

var recursiveFunc = function(promises, allArtists, next) {

var req = {
    method: 'GET',
    url: next || 'https://api.spotify.com/v1/me/following',
    headers: _authConfig,
    params: _params
};

return $http(req)
    .then(function(res) {

        var artistTotal = res.data.artists.total;

        for(var i=0; i<res.data.artists.items; i++) {
            allArtists.push(res.data.artists.items[i]);
        }

        if (allArtists.length < artistTotal) {
            promises.push(recursiveFunc(res.data.artists.next));
        }
    });
}

var promises = [];
var allArtists = [];

recursiveFunc(promises, allArtists);

$q.all(promises).then(function() {
    console.log('All Artists', allArtists);
});

OLD SOLUTION

UPDATE: This answer is not relevant anymore as OP changed the question from "concurrent" requests to "procedural" requests.

After a quick walk through Spotify's API details, it seems a concurrent solution to your problem is not possible. Spotify uses a cursor based paging where each HTTP response contains the cursor to be used for retrieving next set of items. I believe this is sort of a defense mechanism they've chosen against a possible denial of service attack.

https://developer.spotify.com/web-api/object-model/#cursor-based-paging-object

I think the only option will be very similar to what you're already using, i.e. chaining multiple promises together till you retrieve all the artists.

EDIT: Replaced parallel with concurrent to avoid confusion. :)

I am not looking for a parallel solution. "concurrent" requests meaning sending one request after the other completes until we have loaded all artists the user follows — Connor Leech, Nov 16 '15 at 20:15
By definition, concurrent means "existing, happening, or done at the same time." I don't think the answer must be down-voted on that pretext. — Digitrance, Nov 16 '15 at 20:17
oh my bad. I misunderstood (changed question to one after other) — Connor Leech, Nov 16 '15 at 20:18
No worries. I updated my answer to avoid any further confusion. :) — Digitrance, Nov 16 '15 at 20:22
I've updated the answer with a recursive solution. I think this approach should simplify what you're trying to achieve. Good luck, and let me know if it works! — Digitrance, Nov 16 '15 at 20:37

score 0 · Answer 2 · answered Nov 17 '15 at 03:22

Here's one solution to handle this. Use a single promise to let your application know when the retrieval of all artists is complete. Each iteration of the function will make a request to the next url (starting out with the intial spotify api call), attach the items it receives, and then either call itself if more items are needed, or resolve the promise with the built array if not.

This is just a quick example and you might need to modify for your uses. Something like the following:

function getAllArtists() {
  var deferred = $q.defer();
  var allArtists = [];

  var getArtists = function (url) {
    var req = { 
      // request details
      method: 'GET',
      url: url
      // ... other req options
    };

    $http(req).then(function (res) {
      var artistTotal = res.data.artists.total;
      allArtists = allArtists.concat(res.data.artists.items);

      if(allArtists.length < artistTotal) {
        // need more artists
        getArtists(res.data.artists.next);
      } else {
        // we have all artists
        deferred.resolve(allArtists);
      }
    }
  }

  getArtists('https://api.spotify.com/v1/me/following');

  return deferred.promise;
}

how do I send procedural requests (one after the other) with loop from angular factory?

2 Answers2

UPDATED SOLUTION

OLD SOLUTION