$q, $http and parallel processing of results

Question

I am using Angular's $q to execute multiple $http post calls asynchronously. This is working nicely and the calls are all returning at roughly the same time, within a few milliseconds.

The problem is that the next step is to process the results. I want the results to be processed in parallel but they seem to be queuing up and process one at a time.

I've pasted the code below and would appreciate it if someone can point me in the right direction as to how to ensure the results are processed in parallel. This code is basically the SearchService and its 'search' method is the entry point for the controller.

var searchService = {
    searchOptions: {},
    searchResults: {
        foos: [],
        bars: [],
        typeBs: [],
        typeAs: []
    },
    searchFacets: {
        foos: [],
        bars: [],
        typeBs: [],
        typeAs: []
    },
    processResults: function (object, start) {
        return function (response) {

            //var marker = +new Date();  // log end timestamp
            //var diff = marker - start;
            //console.log("start processing " + object.name + ": " + diff);

            ...cut for brevity...

            //marker = +new Date();  // log end timestamp
            //diff = marker - start;
            //console.log("finished processing " + object.name + ": " + diff);
        }
    },
    getSingleSearchParams: function (object) {
        return {
            searchPhrase: this.searchOptions.searchPhrase,
            type: object.typeId,
            facets: object.facets
        };
    },
    getSearchResults: function (object, start) {
        if (object.include)
            dataFactory.searchFactory.search(this.getSingleSearchParams(object)).success((this.processResults)(object, start));
        else {
            var message = object.name + " not selected for search";
            throw message;
        }
    },
    search: function () {
        var start = +new Date();
        var searches = {};
        if (this.searchOptions.foos.include) {
            searches.fooResults = this.getSearchResults(this.searchOptions.foos, start);
        }
        if (this.searchOptions.bars.include) {
            searches.barResults = this.getSearchResults(this.searchOptions.bars, start);
        }
        if (this.searchOptions.typeBs.include) {
            searches.typeBResults = this.getSearchResults(this.searchOptions.typeBs, start);
        }
        if (this.searchOptions.typeAs.include) {
            searches.typeAResults = this.getSearchResults(this.searchOptions.typeAs, start);
        }
        $q.all(searches);
    }
};

SearchFactory's 'search' method contains the $http calls:

searchFactory.search = function (searchOptions) {
    return $http.post(searchUrl, searchOptions);
};

The commented out code shows the timings and this is revealed the serial nature of processing the results...

start processing TypeBs: 162
finished processing TypeBs: 164
start processing TypeAs: 535
finished processing TypeAs: 535
start processing Foos: 553
finished processing Foos: 554
start processing Bars: 1616
finished processing Bars: 1617

use $timeout() for each processing. It will be run in parallel — Joao Polo, Jul 14 '15 at 22:44
I'm using the success function already - that's where processResults() is called from — Drammy, Jul 14 '15 at 22:47
or is it because I'm trying to pass object (and start) in as parameters and therefore have processResults() return a function? — Drammy, Jul 14 '15 at 22:50
I suspected that might be the case. So how do I implement webworkers into my code? — Drammy, Jul 14 '15 at 22:57
This [SO answer](http://stackoverflow.com/questions/21310964/angularjs-q-all) might be useful. I think you want to use `$q.all()`. — Dr G., Jul 14 '15 at 22:57
I was just pointing out the SO answer that uses a loop, which you are not using. — Dr G., Jul 14 '15 at 23:19
@Drammy, I think your code is correct. But $q.all() is used with promise objects. It's not parallel because your getSearchResults() doesn't return a promise object, and then, it's executed one by one. — Joao Polo, Jul 15 '15 at 00:07

bioball · Accepted Answer · 2015-07-16T03:49:57.667

Your HTTP requests are running in parallel. Your processResults function is executed after your HTTP requests have returned, which is why they appear serial to you. Add a log line here and see what happens:

getSearchResults: function (object, start) {
    if (object.include)
        console.log("starting request for object %o", object); 
        // this is where the HTTP request is made
        dataFactory.searchFactory.search(this.getSingleSearchParams(object)).success((this.processResults)(object, start));
    else {
        var message = object.name + " not selected for search";
        throw message;
    }
},

By the way, $q.all doesn't do anything for you in your code. The benefit of $q.all is that it allows you to do something after guaranteeing that all of your promises have resolved. However, you're processing each request as soon as it returns.

var promises = [
  $http.get('foo'), 
  $http.get('bar')
];

// At this point, both the HTTP requests have already fired, in parallel. $q.all isn't responsible for this.

$q.all(promises)
.then(function(responses){
  // If you to execute some logic that can only happen after both HTTP requests return, this is where you'd use $q.all.
});

If you want to process your search results only after all the searches have finished, then place your processing business logic inside the callback for $q.all.

EDIT: I refactored your code a little bit so it uses $q.all.

var searchService = {
  processResults: function(responses) {
    // responses is now an array of response objects. Do whatever you will with it here.
  },
  getSingleSearchParams: function(object) {
    return {
      searchPhrase: this.searchOptions.searchPhrase,
      type: object.typeId,
      facets: object.facets
    };
  },
  getSearchResults: function(object, start) {
    // No need to throw an error here. This is the only place you call this method, so you can guarantee that it was called with an object that was selected to search.
    // I've also added a `return` keyword, that actually returns a promise, and removed `processResults`. Process the reults after `$q.all`.
    return dataFactory.searchFactory.search(this.getSingleSearchParams(object));
  },
  search: function() {
    var self = this;
    var start = new Date();
    var promises = ['foos', 'bars', 'typeBs', 'typeAs'].reduce(function(promises, search) {
      if (self.searchOptions[search].include) {
        promises.push(self.getSearchResults(self.searchOptions[search], start))
      }
      return promises;
    }, []);
    return $q.all(promises)
    .then(this.processResults.bind(this));
  }
};

Thanks - I already worked out that the processResults method is running in serial - how do I make them run in parallel? — Drammy, Jul 15 '15 at 08:45
I was using $q.all to execute multiple $http requests simultaneously. I assume from your comment that there is another way to do this? — Drammy, Jul 15 '15 at 08:45
Drammy - you are passing in the searches object to $q.all(), which expects an array of promises. Even the properties on searches aren't promises (because getSearchResults doesn't return a promise), so I don't see how it can work the way you currently use it. — lintmouse, Jul 15 '15 at 14:55
OK, I'll change the searches object so the properties are promises. I'm not sure this will make any difference though as its working as I expect bar the serial processing of the results. — Drammy, Jul 15 '15 at 21:22
Just made an edit that refactors your code. Does that clear anything up for you? — bioball, Jul 16 '15 at 03:50
That's great thanks bioball. I'm still learning js (as you can probably tell) - there's a few bits you've done that are new to me - off to read up... Thanks again — Drammy, Jul 17 '15 at 10:50

$q, $http and parallel processing of results

1 Answers1