1

The question is marked duplicate wrongly. It is a different situation all together.

I have a database containing roughly 27000 users. I want to insert more users to it from a csv file. Basically each user can uniquely identified by the email Id. I want to enter 43000 more such emails to the database. I am planning to use

Users.insertMany(emailArray,options,callback)

I want that if the email id already exists, it should not enter the record to the database. Checking for each email id in the database and updating individually seems inefficient. insertMany seems faster. Can I work my way around using insertMany?

Current Approach::

csv()
    .fromFile(path)
    .on('json', (jsonObj) => {
    var email = jsonObj.email;
    delete jsonObj.email;
    var profile = jsonObj;
    User.addEmails(email, profile, function (err, res) {
        if (err == "User already exists") {
            console.log("user already exists");
            } else {
            console.log(res);
            }
        })
    })
    .on('done', (error) => {
        console.log('end');
        fs.unlink(path);
        callback(null, "Done");
    })

Add emails function

module.addEmails = function(email, profile, callback){
    User.findOne({email: email}, function(err, res){
        if(err)
            throw err;
        else if(res){
            callback("User already exists");
        }
        else{
            User.create({email: email, profile: profile},callback);             
        }
    })
}
Ankur Chavda
  • 33
  • 11
  • Those are very bad duplicate examples... One is python, and definitely does not use mongoose and the other is for one document only, not bulk. I advice you to use updateMany (introduced in v3.2) with the upsert option. http://mongoosejs.com/docs/api.html#model_Model.updateMany & https://docs.mongodb.com/manual/reference/method/db.collection.updateMany/#db.collection.updateMany – Nico Van Belle Sep 12 '17 at 09:04
  • @NicoVanBelle So using upsert would replace the existing doc right? In that case, I also have some fields in the doc that cannot be removed else would lead to major data loss – Ankur Chavda Sep 12 '17 at 09:30

0 Answers0