Javascript how to execute code after for loop completes

Question

I'm trying to work through this js/async scenario and i'm trying to know how the rest of the js world handles this.

function doStuff(callback) {

  cursor.each(function(err, blahblah) {
    ...doing stuff here takes some time
  });

  ... Execute this code ONLY after the `cursor.each` loop is finished
  callback();

EDIT

Here's a more concrete example updated using most of the suggestions below which still doesn't work.

function doStuff(callback) {

  MongoClient.connect(constants.mongoUrl, function(err, db) {

    var collection = db.collection('cases2');
    var cursor = collection.find();

    var promises = [];  // array for storing promises

    cursor.each(function(err, item) {

      console.log('inside each'); // NEVER GETS LOGGED UNLESS I COMMENT OUT THIS LINE: return Q.all(promises).then(callback(null, items));

      var def = Q.defer();        // Create deferred object and store
      promises.push(def.promise); // Its promise in the array

      if(item == null) {
        return def.resolve();
      }

      def.resolve();  // resolve the promise
    });

    console.log('items'); // ALWAYS GETS CALLED
    console.log(items);

    // IF I COMMENT THIS LINE OUT COMPLETELY, 
    // THE LOG STATEMENT INSIDE CURSOR.EACH ACTUALLY GETS LOGGED
    return Q.all(promises).then(callback(null, items));
  });
}

@VivinPaliath I think that part is irrelevant. I just loop through values of blah and push to an array. — Catfish, Jul 31 '14 at 16:21
Might be helpful: http://stackoverflow.com/questions/16026942/how-do-i-chain-three-asynchronous-calls-using-jquery-promises — emerson.marini, Jul 31 '14 at 16:22
How would I know when i'm at the last element of `cursor.each`. There is no index param from what I can tell http://mongodb.github.io/node-mongodb-native/api-generated/cursor.html#each — Catfish, Jul 31 '14 at 16:26
@Catfish I'd look here - https://github.com/kriskowal/q - plenty of examples on how to wait for multiple async things to finish before proceeding. I presume the async Mongo API you're calling within `cursor.each` also has the ability to invoke a callback at the end of each individual action? — Alnitak, Jul 31 '14 at 16:26
I'm not following you 100%. What do you mean by `I presume the async Mongo API you're calling within cursor.each also has the ability to invoke a callback for each individual action?` — Catfish, Jul 31 '14 at 16:30
The behaviour you now describe is puzzling - however your `.then` callback is incorrect because you're passing the _result_ of calling `callback(null, items)` to `.then` instead of just passing a _reference_ to the function referred to by `callback` — Alnitak, Jul 31 '14 at 18:55
Also, I presume that `MongoClient.connect` is _itself_ async, such that `function(err, db)` is called on completion of the connection and `doStuff()` then returns immediately? — Alnitak, Jul 31 '14 at 18:56
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/58446/discussion-between-catfish-and-alnitak). — Catfish, Jul 31 '14 at 19:34

dreamlab · Answer 1 · 2014-07-31T20:47:42.423

5

without using promises or any other dependencies/libraries you can simply

function doStuff(callback) {

add a counter

    var cursor = new Array(); // init with some array data
    var cursorTasks = cursor.length;

    function cursorTaskComplete()
    {
        cursorTasks--;

        if ( cursorTasks <= 0 ) {
            // this gets get called after each task reported to be complete
            callback();
        }
    }

    for ( var i = 0; i < cursor.length; i++ ) {
        ...doing stuff here takes some time and does some async stuff

check after each async request

        ...when async operation is complete call
        cursorTaskComplete()
  }
}

edited Jul 31 '14 at 20:47

answered Jul 31 '14 at 16:44

dreamlab

3,321
20
23

1

This is the way that I thought about doing it without promises. – Catfish Jul 31 '14 at 17:24
it definitively safes a lot of processing and has less complexity. – dreamlab Jul 31 '14 at 17:35
There's no `cursor.length` property - there's a `cursor.count()` method but that is itself async! – Alnitak Jul 31 '14 at 20:28
at the time of writing there was a just an array in the question. this solution solves a general case as referred to before EDIT. – dreamlab Jul 31 '14 at 20:42

Alnitak · Accepted Answer · 2014-07-31T19:57:57.140

3

Without knowing the details of the async calls you're making within the cursor.each loop, I shall assume that you have the ability to invoke a callback each time the functions invoked therein have completed their async task:

function doStuff() {
    var promises = [];  // array for storing promises

    cursor.each(function(err, blahblah) {
        var def = Q.defer();        // create deferred object and store
        promises.push(def.promise); // its promise in the array

        call_async_function(..., def.resolve);  // resolve the promise in the async function's callback
    });

    // pass the array to Q.all, only when all are resolved will "callback" be called
    return Q.all(promises);
}

and the usage then becomes:

doStuff().then(callback)

Note how the invocation of the callback now never touches the doStuff function - that function now also returns a promise. You can now register multiple callbacks, failure callbacks, etc, all without modifying doStuff. This is called "separation of concerns".

[NB: all the above based on the Q promises library - https://github.com/kriskowal/q]

EDIT further discussion and experimentation has determined that the .each call is itself async, and gives no indication to the outside when the last row has been seen. I've created a Gist that demonstrates a resolution to this problem.

edited Jul 31 '14 at 19:57

answered Jul 31 '14 at 16:35

Alnitak

334,560
70
407
495

Sorry i guess i'm NOT actually doing async stuff in my `cursor.each` loop. It's just the cursor.each loop that is not completing before my callback method. Not sure if that affects your answer or not. – Catfish Jul 31 '14 at 16:39
This doesn't seem to work. `return Q.all(promises).then(callback);` is still being executed before anything inside `cursor.each is executed`. – Catfish Jul 31 '14 at 16:57
I think the problem is that it takes a while to hit the callback of `cursor.each` so `return Q.all(promises).then(callback);` executes before `cursor.each` and `promises` is an empty array so it just fires the callback. I'm not sure how to circumvent this with promises. – Catfish Jul 31 '14 at 17:04
No, in theory the `promises` array will already be full (because `.each` isn't async) by the time `Q.all` is called. – Alnitak Jul 31 '14 at 17:47
@Catfish re: your first comment - if there was no async code in the `cursor.each` loop then your original code would have worked. Re: your second comment - yes, that's how promises work. The `Q.all` and `.then` functions are themselves async - they return immediately. You can't "busy idle" in JS so you _must_ have some mechanism to not only wait for the async stuff to finish, but to wait for `doStuff` to finish too. Promises provide that. – Alnitak Jul 31 '14 at 18:46
But my original code doesn't work and your example doesn't work either. Please see my EDIT to the question. I hope that makes things clearer – Catfish Jul 31 '14 at 18:51

score 0 · Answer 3 · answered Jul 31 '14 at 17:07

if you want to do it with the async module, you can make use of the async forEachSeries function

Code snippet:

function doStuff(callback) {

  async.forEachSeries(cursor, function(cursorSingleObj,callbackFromForEach){
      //...do stuff which takes time
      //this callback is to tell when everything gets over execute the next function
      callbackFromForEach();
  },function(){
     //over here the execution of forEach gets over and then the main callback is called
    callback();
  });
}

bhantol · Answer 4 · 2014-07-31T19:11:59.743

-1

In my mind an elegant/ideal solution would be to have something like

 cursor.each(........).then( function() { ....your stuff});

But without that you can do this....UPDATED

http://plnkr.co/edit/27l7t5VLszBIW9eFW4Ip?p=preview

The gist of this is as shown below...notice....when

var doStuff = function(callback) {
      cursor.forEach(function(cursorStep) {
        var deferred = $q.defer();
        var promise = deferred.promise;
        allMyAsyncPromises.push(promise);
        cursorStep.execFn(cursorStep.stepMeta);
        promise.resolve;
      });

      $q.when(allMyAsyncPromises).then(callback);
}

After hitting the start button wait for few seconds...the async tasks have been simulated to finish in 5 seconds so the status will update accordingly.

Not having access to a real cursor object..I had to resort of fake cursor like and array.

edited Jul 31 '14 at 19:11

answered Jul 31 '14 at 17:05

bhantol

9,368
7
44
81

Your second solution is basically the same as @Alnitak's and the problem it doesn't work. It's returning because the code even gets inside of `cursor.each`. – Catfish Jul 31 '14 at 17:24
Yes similiar to @Alnitak but the minor difference is in the "starting to run" vs "completeing the async step". I created a working plnkr for you. Notice that the status "starting to run" is in the callback. http://plnkr.co/edit/27l7t5VLszBIW9eFW4Ip?p=preview – bhantol Jul 31 '14 at 19:14

Javascript how to execute code after for loop completes

4 Answers4

Linked