Sequelize transaction retry doens't work as expected

Question

I don't understand how transaction retry works in sequelize. I am using managed transaction, though I also tried with unmanaged with same outcome

await sequelize.transaction({ isolationLevel: Sequelize.Transaction.ISOLATION_LEVELS.REPEATABLE_READ}, async (t) => {

    user = await User.findOne({
        where: { id: authenticatedUser.id },
        transaction: t,
        lock: t.LOCK.UPDATE,
    });    

    user.activationCodeCreatedAt = new Date();
    user.activationCode = activationCode;
    await user.save({transaction: t});
});

Now if I run this when the row is already locked, I am getting

DatabaseError [SequelizeDatabaseError]: could not serialize access due to concurrent update

which is normal. This is my retry configuration:

retry: {
    match: [
        /concurrent update/,
    ],
    max: 5
}

I want at this point sequelize to retry this transaction. But instead I see that right after SELECT... FOR UPDATE it's calling again SELECT... FOR UPDATE. This is causing another error

DatabaseError [SequelizeDatabaseError]: current transaction is aborted, commands ignored until end of transaction block

How to use sequelizes internal retry mechanism to retry the whole transaction?

Unfortunately it doesn't support transaction retry so I ended up implementing my own based on async-retry — Pablo, Aug 26 '21 at 21:54

Ciro Santilli OurBigBook.com · Answer 1 · 2021-12-04T15:59:34.287

Manual retry workaround function

Since Sequelize devs simply aren't interested in patching this for some reason after many years, here's my workaround:

async function transactionWithRetry(sequelize, transactionArgs, cb) {
  let done = false
  while (!done) {
    try {
      await sequelize.transaction(transactionArgs, cb)
      done = true
    } catch (e) {
      if (
        sequelize.options.dialect === 'postgres' &&
        e instanceof Sequelize.DatabaseError &&
        e.original.code === '40001'
      ) {
        await sequelize.query(`ROLLBACK`)
      } else {
        // Error that we don't know how to handle.
        throw e;
      }
    }
  }
}

Sample usage:

const { Transaction } = require('sequelize');

await transactionWithRetry(sequelize,
  { isolationLevel: Transaction.ISOLATION_LEVELS.SERIALIZABLE },
  async t => {
    const rows = await sequelize.models.MyInt.findAll({ transaction: t })
    await sequelize.models.MyInt.update({ i: newI }, { where: {}, transaction: t })
  }
)

The error code 40001 is documented at: https://www.postgresql.org/docs/13/errcodes-appendix.html and it's the only one I've managed to observe so far on Serialization failures: What are the conditions for encountering a serialization failure? Let me know if you find any others that should be auto looped and I'll patch them in.

Here's a full runnable test for it which seems to indicate that it is working fine: https://github.com/cirosantilli/cirosantilli.github.io/blob/dbb2ec61bdee17d42fe7e915823df37c4af2da25/sequelize/parallel_select_and_update.js

Tested on:

    "pg": "8.5.1",
    "pg-hstore": "2.3.3",
    "sequelize": "6.5.1",

PostgreSQL 13.5, Ubuntu 21.10.

Infinite list of related requests

https://github.com/sequelize/sequelize/issues/1478 from 2014. Original issue was MySQL but thread diverged everywhere.
https://github.com/sequelize/sequelize/issues/8294 from 2017. Also asked on Stack Overflow, but got Tumbleweed badge and the question appears to have been auto deleted, can't find it on search. Mentions MySQL. Is a bit of a mess, as it also includes connection errors, which are not clear retries such as PostgreSQL serialization failures.
https://github.com/sequelize/sequelize/issues/12608 mentions Postgres
https://github.com/sequelize/sequelize/issues/13380 by the OP of this question

Meaning of current transaction is aborted, commands ignored until end of transaction block

The error is pretty explicit, but just to clarify to other PostgreSQL newbies: in PostgreSQL, when you get a failure in the middle of a transaction, Postgres just auto-errors any following queries until a ROLLBACK or COMMIT happens and ends the transaction.

The DB client code is then supposed to notice that just re-run the transaction.

These errors are therefore benign, and ideally Sequelize should not raise on them. Those errors are actually expected when using ISOLATION LEVEL SERIALIZABLE and ISOLATION LEVEL REPEATABLE READ, and prevent concurrent errors from happening.

But unfortunately sequelize does raise them just like any other errors, so it is inevitable for our workaround to have a while/try/catch loop.

Sequelize transaction retry doens't work as expected

1 Answers1