Non-recursive method to iterate over Promise iterator

Question

I've developed a client library that exposes a method called iterator(). This method returns a Promise instance created using require('promise') library, which is completed with an iterator object.

This object contains a method called next() which returns a Promise which is completed with a complex object like this: {done: [true|false], key: _, value: _}

Although iterator() might pre-fetch some elements, next() needs to return a Promise in case it results in a remote call.

Now, say a user wants to iterate over all elements until the Promise returned by next() returns an object containing done: true.

I've managed to achieve this using the following recursive method (I originally found this solution in this answer):

var iterate = client.iterator();

iterateTeams.then(function(it) {

  function loop(promise, fn) {
    // Simple recursive loop over iterator's next() call
    return promise.then(fn).then(function (entry) {
      return !entry.done ? loop(it.next(), fn) : entry;
    });
  }

  return loop(it.next(), function (entry) {
    console.log('entry is: ' + entry);
    return entry;
  });

});

The question is, would it be possible, using require('promise') library, to build a non-recursive solution? The reason I'm interested in a non-recursive method is to avoid blowing up if the number of entries to iterate over is too big.

Cheers, Galder

It sounds like you're looking for a generator or observable. — ssube, Apr 01 '16 at 17:01
you can use another solution which _hides_ the recursion for you but it still would be a recursive solution — smnbbrv, Apr 01 '16 at 17:03
Galder, you probably want to read the section entitled "The Collection Kerfuffle" [here](http://taoofcode.net/promise-anti-patterns/) — Roamer-1888, Apr 02 '16 at 08:40
@Roamer-1888, thanks for the pointer but that section assumes you have an array or know how many elements you need to loop through and hence does not apply here. — Galder Zamarreño, Apr 04 '16 at 08:53
[There is no reason why this would blow up](http://stackoverflow.com/q/29925948/1048572). — Bergi, Apr 04 '16 at 09:23

score 3 · Accepted Answer · edited May 23 '17 at 11:53

The reason I'm interested in a non-recursive method is to avoid blowing up if the number of entries to iterate over is too big

Don't fear. Asynchronous "recursion" (sometimes dubbed pseudo-recursion) does not grow the call stack, it's much like tail recursion. You won't ever get a stackoverflow exception.

And if the promise library is implemented reasonably, this should not even grow the memory - see Building a promise chain recursively in javascript - memory considerations for details.

Benjamin Gruenbaum · Answer 2 · 2016-04-04T08:53:26.687

1

Without new syntax or a library - generally no.

Well, if you're using babel, you can use ES2018 (:P) async iteration:

for await (const team of iterateTeams) {
   // do something with team
}

read more about it here

Otherwise, you can use generators with ES2016 async/await syntax:

for(var it = iterateTeams(); !done; ({done, value}) = await it.next()) {
    // work with value
}

Or with available today ES2015 generator syntax and a pump through bluebird:

// inside a Promise.corutine  
for(var it = iterateTeams(); !done; ({done, value}) = yield it.next()) {
   // work with value
}

edited Apr 04 '16 at 08:53

answered Apr 02 '16 at 13:31

Benjamin Gruenbaum

270,886
87
504
504

Thanks a lot for those solutions but none of those work for me. I'm using ES2015 and require Node.js 0.10 (bluebird Promise.coroutine requires Node.js 0.12). – Galder Zamarreño Apr 04 '16 at 08:52
@GalderZamarreño how are you using ES2015 if you're using NodeJS 0.10? – Benjamin Gruenbaum Apr 04 '16 at 08:53
1

Also, the answer still stands, without recursion or a helper method that does recursion itself - it's impossible before ES2015 – Benjamin Gruenbaum Apr 04 '16 at 08:53
Sorry, got confused, mean ES5 and Node.js 0.10. – Galder Zamarreño Apr 04 '16 at 08:55
Then no, you have to use recursion or a helper function that would use recursion itself - effectively extracting your `loop` function to a helper. – Benjamin Gruenbaum Apr 04 '16 at 09:18

Non-recursive method to iterate over Promise iterator

2 Answers2

Linked