2

I am writing a Node.js script to populate an SQL database with a test dataset.

In the promise chain illustrated in the code snippet below (the real code is a bit more hairy), the function insertData() requires the db object to be passed through from the previous stage. However, asynchronous calls inside dropAndCreateTables() on the previous stage use db object but do not return it. I came up with a solution that wraps promises inside dropAndCreateTables() into another promise object that resolves to a db object. However:

  • I heard that using Promise() constructor in non-library code is an antipattern and may lead to subtle and hard-to-diagnose mistakes
  • I heard that nesting then()-chains is also an antipattern
  • It does not allow me to ignore errors from the promiseDrop (for example, I don't care if tables don't exist on drop)
  • It is ugly

Questions:

  • Is there a simpler, nicer and more socially accepted way to override the return value of a promise? (in this case, created with Promise.all())
  • Is there a way to restructure my code in a way that this problem does not occur? (That is, I don't exclude the possibility of "XY problem" here)

Code:

const dropAndCreateTables = (db, startClean) => {
  if(startClean) {
    const sqlDrop = fs.readFileSync('drop.sql').toString()
    const promiseDrop = db.raw(sqlDrop)

    const sqlCreate = fs.readFileSync('create.sql').toString()
    const promiseCreate = db.raw(sqlCreate)

    /********* Problems here? ************************************/
    return new Promise((resolve, reject) => {      // Ew?
      Promise.all([promiseDrop, promiseCreate])
        .then(() => {
          resolve(db) // Override the returned value
        })
        .catch(reject)
    })

  }
  return Promise.resolve(db)
}

initDB({ debug: false })
  .then((db) => {
    return dropAndCreateTables(db, START_CLEAN) // Without my hack this does not return `db`
  })
  .then((db) => {
    return insertData(db, DO_UPSERT)  // This needs the `db` object
  })
  .then(() => {
    console.info(`\n${timestamp()} done`)
  })
  .catch(handleError)
Ivan Aksamentov - Drop
  • 12,860
  • 3
  • 34
  • 61

4 Answers4

4

(Some fairly important notes midway and later in the answer, please do read all the way to the end.)

Is there a simpler, nicer and more socially accepted way to override the return value of a promise? (in this case, created with Promise.all())

Yes, you simply return a value from the then handler, and return the promise then returns:

return Promise.all([promiseDrop, promiseCreate])
    .then(() => db);

then (and catch) create promise chains. Each link in the chain can transform the result. then and catch return a new promise that will be fulfilled or rejected based on what happens in their callback:

  • If their callback throws, the promise rejects with the error thrown
  • If their callback returns a non-thenable value (e.g., promise), the promise is fulfilled with that value
  • If their callback returns a thenable value, the promise is resolved to that thenable — it waits for the other promise to settle, then settles the same way

(If the term "thenable" isn't familiar, or you're not clear on the distinction between "fulfill" and "resolve," I go into promise terminology in this post on my blog.)

I heard that using Promise() constructor in non-library code is an antipattern and may lead to subtle and hard-to-diagnose mistakes

The distinction isn't library code vs. non-library code, it's between code that doesn't already have a promise to work with and code that does. If you already have a promise to work with, you almost never want to use new Promise. More: What is the explicit promise construction antipattern and how do I avoid it?

I heard that nesting then()-chains is also an antipattern

You almost never need to nest then chains, because again each link in the chain already has the means of tranforming the result passing through it. So:

// Unnecessary nesting
doSomething()
    .then(a => {
        return doSomethingElse(a * 2)
            .then(b => b * 3);
    })
    .catch(e => { /*...handle error...*/ });

can be more idiomatically and simply written:

doSomething()
    .then(a => doSomethingElse(a * 2))
    .then(b => b * 3);
    .catch(e => { /*...handle error...*/ });

Is there a way to restructure my code in a way that this problem does not occur? (That is, I don't exclude the possibility of "XY problem" here)

Not an X/Y per se, but you have a problem in that code: There's no guarantee the drop will happen before the create! So instead of starting both and letting them run in parallel and watching for the results with Promise.all, ensure those operations happen in sequence:

// Fairly minimal changes
const dropAndCreateTables = (db, startClean) => {
  if(startClean) {
    const sqlDrop = fs.readFileSync('drop.sql').toString()
    return db.raw(sqlDrop)
      .then(() => {
        const sqlCreate = fs.readFileSync('create.sql').toString()
        return db.raw(sqlCreate);
      })
      .then(() => db);
  }
  return Promise.resolve(db)
}

But, I wouldn't use sync file I/O. Instead

const promisify = require("utils").promisify;
const readWithPromise = promisify(fs.readFile);

and then

const dropAndCreateTables = (db, startClean) => {
  if(startClean) {
    const getDrop = readWithPromise('drop.sql');       // Start this first
    const getCreate = readWithPromise('create.sql');   // Then start this
    return getDrop
      .then(dropSql => db.raw(dropSql))                // Got the drop SQL, run it
      .then(() => getCreate)                           // Make sure we have the create SQl
      .then(createSql => db.raw(createSql))            // Run it
      .then(() => db);
  }
  return Promise.resolve(db)
}

Note how we avoid ever busy-waiting on I/O, and we can overlap the DB's drop operation with reading the create SQL.

T.J. Crowder
  • 1,031,962
  • 187
  • 1,923
  • 1,875
  • 1
    `You almost never need to nest then chains` -> IMO this is one of the rare usecases – Jonas Wilms Jan 21 '18 at 14:21
  • Thank you, I learned a lot from your answer! About nested chains: we both came up with `dropAndCreateTables()` that has a chain inside it, which is nested inside the main chain in the global scope. Isn't resulting `dropAndCreateTables()` is similar to the `a => {}` function in your "Unnecessary nesting" example? We cannot refactor the chain out of the `dropAndCreateTables()`. Hence, my confusion with chain nesting guidelines. Could you comment on that? – Ivan Aksamentov - Drop Jan 21 '18 at 14:33
  • @Drop: You mean the "Minimal changes" code? No, that's not *promise* nesting. That's promise chaining. It's fine (and common) to *initiate* a promise in a `then` handler if you have to, which is what that code does. – T.J. Crowder Jan 21 '18 at 14:37
  • @JonasW.: Why? I don't see any benefit to it here at all. – T.J. Crowder Jan 21 '18 at 14:37
  • @T.J.Crowder No, I meant another thing. If I replace the call to `dropAndCreateTables()` by the contents of it (any version), wouldn't it become the same pattern as illustrated by `return doSomethingElse(a * 2).then(b => b * 3);` in the "Unnecessary nesting" example? I fact I tried to do eliminate the call and then to eliminate nesting, but I was uncertain what to do with that `if` statement. Or is it not the kind of nesting I should eliminate? (I have tons more of this kind of nesting in other, more complicated functions) – Ivan Aksamentov - Drop Jan 21 '18 at 14:49
  • @Drop: I'm sorry, I don't follow what you mean there. – T.J. Crowder Jan 21 '18 at 14:55
  • @T.J.Crowder Sorry for not making myself clear. Another attempt: if we replace the call to function `dropAndCreateTables()` with that functions' body (for example, [like this](https://pastebin.com/7QnExhmR)), will resulting nested structure become similar to the example marked by a comment "Unnecessary nesting" in your answer? – Ivan Aksamentov - Drop Jan 21 '18 at 15:18
  • @Drop: Don't assume the problem was at your end. ;-D Yes, it looks similar, but the reason for that is you have two distinct branches in there thanks to the `if (startClearn)`, so you have to have two chains, not just one. That's one of the few times you might need nesting. Although at that point, you should do exactly what you've done: extract the logic in to a smaller piece -- `dropAndCreateTables` -- that you can call. Smaller pieces are easier to comprehend, and easier to combine. :-) – T.J. Crowder Jan 21 '18 at 15:23
  • @T.J.Crowder Thank you so much, especially for mentioning `promisify`. It just has changed my life completely. =) – Ivan Aksamentov - Drop Jan 21 '18 at 15:41
  • @Drop: LOL Also check out this npm module: [promisify](https://www.npmjs.com/package/promisify). Instead of working at the individual function level, it works at the full API level. – T.J. Crowder Jan 21 '18 at 15:44
1

You don't need to call the Promise constructor when returning another promise, you can just write it like:

return Promise.all([promiseDrop, promiseCreate])
    .then(() => db)
    .catch(error => {
       // handle the error or rethrow it
    })
Tsvetan Ganev
  • 8,246
  • 4
  • 26
  • 43
1

You might omit resolving db from dropAndCreateTables like this:

.then((db) => {
    return dropAndCreateTables(db, START_CLEAN).then(Promise.resolve(db));
})
Bsalex
  • 2,847
  • 1
  • 15
  • 22
1

You should not let dropAndCreateTables return a db promise, there is no real usecase for it. So:

return Promise.all([promiseDrop, promiseCreate]);

is enough. Now the chaining part:

initDB({ debug: false }).then(async (db) => {

  await dropAndCreateTables(db, START_CLEAN);
  await insertData(db, DO_UPSERT);

  console.info(`\n${timestamp()} done`)
}).catch(handleError)
Jonas Wilms
  • 132,000
  • 20
  • 149
  • 151
  • Yes, with `async/await` it is indeed much nicer. I tried this yesterday and converted the entire script to use `async/await`. After I added error checking `try/catch` clauses my script has become absolutely unreadable, so I rolled back to `then()/catch()`. However, converting this particular part might be actually worthwhile. I am currently investigating whether I should wrap EVERY async call into `try/catch` or whether they can be grouped. In the latter case I often get unhanded rejection warnings. – Ivan Aksamentov - Drop Jan 21 '18 at 15:29
  • @drop you just need to catch them once. So you actually only need 1 catch at the highest level. But that depends on your code strucure. – Jonas Wilms Jan 21 '18 at 16:28