0

I have a collection with approximately 271K documents. After a mongoimport from a TSV file, all columns were strings. So I need to do two things:

  1. Trim the strings
  2. Convert some columns from string to numeric

I am using the following function: 

db.hd.find().forEach(function(doc) {

    if (typeof doc.HD === "string") {
        db.hd.update({_id : doc._id}, {$set : {"HD" : parseInt(doc.HD.trim())}});
    }

    if (typeof doc.Ptm === "string") {
        db.hd.update({_id : doc._id}, {$set: { "Ptm" : parseFloat(doc.Ptm.trim())}});
    }

    if (typeof doc.n_Ptm === "string") {
        db.hd.update({_id : doc._id}, {$set: { "n_Ptm" : doc.n_Ptm.trim()}});
    }

    if (typeof doc.Ptg === "string") {
        db.hd.update({_id : doc._id}, { $set : { "Ptg" : parseFloat(doc.Ptg.trim())}});
    }

    if (typeof doc.n_Ptg === "string") {
        db.hd.update({_id : doc._id }, { $set : { "n_Ptg" : doc.n_Ptg.trim()}});
    }

    if (typeof doc.SpT === "string") {
        db.hd.update({_id : doc._id}, { $set : { "SpT" : doc.SpT.trim()}});
    }

    if (typeof doc.Int === "string") {
        db.hd.update({_id : doc._id}, { $set : { "Int" : doc.Int.trim()}});
    }
});

This is a very slow function, obviously because of the collection size and amount of work being performed.

Since I'm a relative newbie at Mongo, is there a faster way to accomplish what I'm doing?

Jason

Jason
  • 3,943
  • 12
  • 64
  • 104
  • Note that as existing answers will show, the actual approach depends on your MongoDB version. MongoDB 4.2 allows all of that logic to actually happen on the server. Just about all other versions ( despite some misstatements in answers ) should essentially employ `bulkWrite()`. Note if you need the latter approach which would still have iteration, it's better to programatically build all the `$set` arguments rather than simply add individual updates for each field. – Neil Lunn Sep 30 '19 at 08:19
  • For the MongoDB 4.2 option, most operators needed are [here](https://docs.mongodb.com/manual/reference/operator/aggregation/#type-expression-operators). And of course [`$trim`](https://docs.mongodb.com/manual/reference/operator/aggregation/trim/). Also the core type conversion operator of [`$convert`](https://docs.mongodb.com/manual/reference/operator/aggregation/convert/) would allow fallback logic if a field was not of the expected type. Might be more useful than the shortcut operators and other conditional expressions. – Neil Lunn Sep 30 '19 at 08:21

0 Answers0