41

I am developing an application where I am using MongoDB as database with Nodejs + Express in application layer, I have two collections, namely

  1. users
  2. transactions

Here i have to update wallet of thousands of users with some amount and if successful create a new document with related info for each transaction, This is My code :

 userModel.update({_id : ObjectId(userId)}, {$inc : {wallet : 500}}, function (err, creditInfo) {
    if(err){
        console.log(err);                            
    }
    if(creditInfo.nModified > 0) {
        newTransModel = new transModel({
            usersId: ObjectId(userId),            
            amount: winAmt,         
            type: 'credit',           
        }); 
        newTransModel.save(function (err, doc) {
            if(err){
                Cb(err); 
            }
        });
    }                            
});

but this solution is not atomic there is always a possibility of user wallet updated with amount but related transaction not created in transactions collection resulting in financial loss.

I have heard that recently MongoDB has added Transactions support in its 4.0 version, I have read the MongoDB docs but couldn't get it to successfully implement it with mongoose in Node.js, can anyone tell me how this above code be reimplemented using the latest Transactions feature of MongoDB which have these functions

Session.startTransaction()
Session.abortTransaction()
Session.commitTransaction()

MongoDB Docs : Click Here

Gausul
  • 265
  • 3
  • 16
Gaurav Kumar
  • 698
  • 1
  • 8
  • 15

1 Answers1

57

with mongoose in Node.js, can anyone tell me how this above code be reimplemented using the latest Transactions feature

To use MongoDB multi-documents transactions support in mongoose you need version greater than v5.2. For example:

npm install mongoose@5.2

Mongoose transactional methods returns a promise rather than a session which would require to use await. See:

For example, altering the example on the resource above and your example, you can try:

const User = mongoose.model('Users', new mongoose.Schema({
  userId: String, wallet: Number
}));
const Transaction = mongoose.model('Transactions', new mongoose.Schema({
  userId: ObjectId, amount: Number, type: String
}));

await updateWallet(userId, 500);

async function updateWallet(userId, amount) {
  const session = await User.startSession();
  session.startTransaction();
  try {
    const opts = { session };
    const A = await User.findOneAndUpdate(
                    { _id: userId }, { $inc: { wallet: amount } }, opts);

    const B = await Transaction(
                    { usersId: userId, amount: amount, type: "credit" })
                    .save(opts);

    await session.commitTransaction();
    session.endSession();
    return true;
  } catch (error) {
    // If an error occurred, abort the whole transaction and
    // undo any changes that might have happened
    await session.abortTransaction();
    session.endSession();
    throw error; 
  }
}

is not atomic there is always a possibility of user wallet updated with amount but related transaction not created in transactions collection resulting in financial loss

You should also consider changing your MongoDB data models. Especially if the two collections are naturally linked. See also Model data for Atomic Operations for more information.

An example model that you could try is Event Sourcing model. Create a transaction entry first as an event, then recalculate the user's wallet balance using aggregation.

For example:

{tranId: 1001, fromUser:800, toUser:99, amount:300, time: Date(..)}
{tranId: 1002, fromUser:77, toUser:99, amount:100, time: Date(..)}

Then introduce a process to calculate the amount for each users per period as a cache depending on requirements (i.e. per 6 hours). You can display the current user's wallet balance by adding:

  • The last cached amount for the user
  • Any transactions for the user occur since the last cached amount. i.e. 0-6 hours ago.
Wan B.
  • 18,367
  • 4
  • 54
  • 71
  • 1
    there are two things about this 1) await updateWallet(userId, 500); <<< this gives an error saying await needs to be in a async function , top level await is not allowed 2) as i am doing this in loop async eachseries i.e updating many users wallet , i am getting writeconflit error of transactions & when i call next iteration, also using a finally block after try catch also doesn't work – Gaurav Kumar Jul 09 '18 at 16:41
  • 2
    This is an example, you'd have to adjust for your own use case. 1) Make sure you create an async function as shown above. 2) You can try performing [bulkWrite()](http://mongoosejs.com/docs/api.html#bulkwrite_bulkWrite) within a transaction instead. – Wan B. Jul 10 '18 at 01:06
  • thanks for assistance, i already implemented this successfully, the second error was my coding mistake not an issue of the code. – Gaurav Kumar Jul 10 '18 at 02:30
  • 1
    just you edit your answer and edit this >> User.db.startSession(); to Mymodel.startSession() ........it says db is undefined, we can use any mongoose model to start session its not scoped to that model once created. – Gaurav Kumar Jul 10 '18 at 02:31
  • Ah yes, I was trying to give emphasis that the session for for the database not on the collection, but made a mistake. Fixed now. thanks. – Wan B. Jul 10 '18 at 07:57
  • question: I don't see any use of .endSession(), could somebody care to explain whether we should need it or not? – Yogesh Sep 23 '20 at 11:01
  • @WanBachtiar how oo i add new: true as well as sessions in 3rd param? – Kannan T Dec 04 '20 at 12:16
  • @Yogesh ..... like in mysql even this works on a session which mongodb creates, eveithing is registered on a session and only written to db when you call committ, its a good practice to end the session when your work is done like we do end connection in mysql, its good for your server resources too & security. – Gaurav Kumar Dec 27 '20 at 12:43
  • Well, in that case, @WanBachtiar, it would a good idea to add a `finally` block to `endSession()`. It would a) save a code line b) would guarantee the execution. – Yogesh Jan 03 '21 at 13:06
  • 1
    @GauravKumar, that comment was supposed to go to Kannan T right ? – Wan B. Jan 05 '21 at 04:54
  • @Yogesh, that would be your choice on how to structure your code. The example above is to show how a transaction based on a session is started and committed/aborted. – Wan B. Jan 05 '21 at 04:56
  • @KannanT ..... i think you can just declare an object in their third parameter something like this { session : session , new : true } – Gaurav Kumar Jan 23 '21 at 08:41
  • @WanBachtiar ...yes it was for kannan T......mentioned you by mistake !! – Gaurav Kumar Jan 23 '21 at 08:42
  • MongooseError: Callback must be a function, got [object Object] I have got this error. What can I do for this – Jonny May 23 '21 at 18:50
  • @WanBachtiar Can we handle concurrent error with transaction ? I mean if I update 2 times to 1 record so fast, and it's updated to the same __v of record so the concurrency updating is happen, if it's throw to catch, can I abort transaction or the concurrent error can happen even transaction is not committed ? – Vũ Anh Dũng Aug 05 '21 at 15:46
  • 1
    Would this still work properly if the two queries would be executed in parallel using await Promise.all? – Anthony Yershov Dec 31 '21 at 05:43
  • Fantastic, thanks ! – jonathasborges1 Jan 03 '22 at 03:14