I'm having issues optimising update performance for a Meteor collection.
I take a document from one collection (CollectionA), modify it, cherry-pick some of the content, and then update a near duplicate of it in another collection (CollectionB). I also upsert in case new documents have been added to CollectionA
Everything happens in a few milliseconds, but the update can take anywhere from 10–30+ seconds depending on the size of the document being updated. The documents are typically around 30kb...
I have tried without Meteor.defer, with writeConcern : 0, and on both local and cloud replica set clusters. Have also tried using insert instead of update. Nothing makes a noticeable difference.
cronTask(){
CollectionA.find({isPublished : true, "approval.status" : { $gte : 1 } }).forEach((doc)=>{
let newDoc = {
parentId : doc._id,
slug : doc.slug, // these never change, only the content array changes...
title : doc.title,
description: doc.description,
tags : doc.tags,
category : doc.category,
createdAt : new Date(),
content: [...,...,...] // the content of the new document is cherrypicked from the parents before saving
}
while(doc.content.length) {
// cherry-picking and pushing to newDoc.content
// super quick, a couple of MS
}
Meteor.defer(()=>{
CollectionB.update({parentId : doc._id}, {$set : newDoc}, {upsert : true}, (err,res)=>{
if (!err) {
console.log(`Saved child for ${doc.title}`);
} else {
console.log(`Error saving child for ${doc.title}: ${err}`);
}
});
});
});
}
