3

I initially assumed that passing a bare Promise.mapSeries(...) call as an argument to .then() would be the same as wrapping it in a function, like .then(function() { return Promise.mapSeries(...); }). Having written out this question, I'm no longer entirely sure why it works at all.

In the simplified code below, I open a couple of databases asynchronously (openDBAsync()) and then read a file containing a JS object. I use _.map() to iterate over all of the key/value pairs in the object and asynchronously update their values in a database, while keeping track of which ones meet certain criteria (whether the value is odd, in this toy example). Promise.all() waits for all of the async database calls to settle, and then Promise.mapSeries() is used to process each of the subset of keys, which makes another async database call for each one. Finally, I close all the databases.

function processData(path)
{
    var oddKeys = [];

    return openDBAsync()
        .then(function() { return readFileAsync(path); })
        .then(function(dataObject) {
            return Promise.all(_.map(dataObject, function(value, key) {
                if (value % 2) {
                    oddKeys.push(key);
                }

                return updateDBAsync(key, ++value);
            }))
            .then(Promise.mapSeries(
                oddKeys,
                function(key) {
                    return updateOddDBAsync(key);
                }
            ))
        })
        .then(closeDBAsync);
}

The problem is that the database throws errors complaining that I'm trying to update the database after it's been closed. That means that some of the promises generated in the .mapSeries() call are being called after the final closeDBAsync(). I expected all of them to settle before the final .then() call.

If I wrap the call to Promise.mapSeries() in a function:

            .then(function() {
                return Promise.mapSeries(
                    oddKeys,
                    function(key) {
                        return updateOddDBAsync(key);
                    }
                );
            })

Then I don't get any errors. It also works if I put a .delay(2000) before the close database call, which indicates that Promise.mapSeries() isn't settling all of the promises before it finishes.

This seems like either a bug in Bluebird, or, much more likely, I'm not understanding something fundamental about how Promise.mapSeries() works. Any guidance would be much appreciated.

jdunning
  • 1,356
  • 1
  • 14
  • 26
  • 1
    Without the function it will run right away when `oddKeys` is not ready to be consumed. It just looks like an anti-pattern to push things to an array within a promise to then make use of it in another promise. Why not thread this data asynchronously? – elclanrs Dec 04 '15 at 17:46
  • `.then(Promise.mapSeries(...))` will invoke `mapSeries` immediately and pass *its result* into `.then(...)` to be called later. Your argument to `mapSeries` is not a promise, so it will map that argument immediately. Follow elclanrs's advice and restructure this without side-effects (e.g. one thenable to just collect the odd keys and another to process them, without keeping an external state variable). – DCoder Dec 04 '15 at 17:52
  • Thanks, @elclanrs and @DCoder. I agree that having the array outside of the promise thread is not ideal. But the direct call to `Promise.mapSeries()` **doesn't** get applied to the array immediately. If it was, the array would be empty and this code wouldn't work at all. Looking at the Bluebird code, there's some logic I don't entirely understand that decides when to start iterating on the array. I'm wondering if something about this logic could get confused about when it's finished mapping the array. – jdunning Dec 04 '15 at 20:34
  • Are you by any chance calling processData multiple times ? That could cause 1 async close db statement to interrupt another processData call. – Willem D'Haeseleer Dec 04 '15 at 21:28
  • I am calling `processData()` multiple times, but in the real code, the calls to open and close the database are called only once, outside of this function. I'd included the calls in this function just to simplify the description. – jdunning Dec 05 '15 at 01:17
  • I tried to build a standalone code snippet to reproduce this issue (which is something you should have provided, actually, for accurate results). It shows that calling `.then(Promise.mapSeries(...))` doesn't work as you think it does, and it even emits a console warning: `.then() only accepts function, but was passed: [object Object]`, because `.mapSeries` returns a promise, which cannot be used as a `.then()` callback. – DCoder Dec 05 '15 at 06:34
  • Thanks, @DCoder. When running the actual code in Node, I didn't see any such warning, which would definitely have been helpful. Do you have to do something special to enable warnings in Bluebird? I see that its `warn()` method maps to `console.log()`, and I see my log messages and Bluebird errors in the console, but don't seem to see its warnings. – jdunning Dec 05 '15 at 22:48

1 Answers1

3

much more likely, I'm not understanding something fundamental about how Promise.mapSeries() works

Nope, this seems to be more a misunderstanding about how .then(…) works.

The then method of promises does always take a callback function (if you're passing anything else [but null], Bluebird should spit out a warning!). By calling .then(Promise.mapSeries(…)), you were passing a promise instead, which is simply ignored. Being ignored, it also is not awaited by anything, which leads to that error of the database being closed too early.

But the direct call to Promise.mapSeries() doesn't get applied to the array immediately. If it was, the array would be empty and this code wouldn't work at all.

Yes it does. Your array is filled from by the _.map callback, which is executed synchronously, before then and mapSeries are invoked.

So the solution is indeed to wrap the call in a function expression, which will only be executed when the Promise.all(…) fulfills, and whose result will then not be ignored but rather awaited. There might be more, different solutions, depending on what degree of parallel execution you want to allow.

Btw, given you are doing a database transaction, your current code is quite fragile. Look into the promise disposer pattern.

Community
  • 1
  • 1
Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • Thanks, @Bergi. Obvious when you point it out that the `_.map()` runs synchronously. I think I was confusing myself because it was wrapped in a `Promise.all()`, which made my brain think "this bit happens later". I'll take another look at disposers, though the documentation was a bit abstruse last time I checked it out. – jdunning Dec 05 '15 at 22:52