0

I am trying to convert a huge set of dates to ISODate objects and storing it in the same collection and field. I am using the following query:

db.collection.find().forEach(function(element){
  element.StartTime = ISODate(element.StartTime);
  element.StopTime = ISODate(element.StopTime);
  db.collection.save(element);
});

The query ran for about 10 minutes, and then gave an error:

2017-06-15T15:48:10.419+0200 E QUERY    [thread1] Error: invalid ISO date :
ISODate@src/mongo/shell/types.js:65:1
@(shell):2:23
DBQuery.prototype.forEach@src/mongo/shell/query.js:501:1
@(shell):1:1

I had a look into the entries in the DB and it looks like it did convert a lot of the data set, but I am having troubles now fixing the error and finding the location where it stopped. What I tried is using Studio3T to find where either "StartTime" or "StopTime" has a value of 2017-06-15T15:48:10.419+0200 or at least starts with 2017-06, but as I expected (since there shouldn't be any data from June 2017 in there), it can't find anything.

Now when I run the query again, it gives the error immediately.

My question is if it's possible to find the document in the collection responsible for this error or what the cause might be. Are there recommendations for bulk operations like this preventing these errors?

ffritz
  • 2,180
  • 1
  • 28
  • 64
  • Umm. `new Date()` and not `ISODate()`. And most importantly don't use `.save()` for large updates. Use updates [more like this](https://stackoverflow.com/a/37280419/2313887) – Neil Lunn Jun 15 '17 at 14:11
  • Also learn about [`.hasOwnProperty()`](https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Object/hasOwnProperty) – Neil Lunn Jun 15 '17 at 14:13
  • @NeilLunn Thanks, will look into. I was using ISODate because it was suggested multiple times, e.g. https://stackoverflow.com/a/15474233/4755172. – ffritz Jun 15 '17 at 14:16
  • Don't believe everything you read on SO. It's mostly lies :) – Neil Lunn Jun 15 '17 at 14:17
  • @NeilLunn Roger that. I came up with this: `db.alltrips.aggregate( [ { "$addFields": { "start": new Date("$StartTime"), "stop": new Date("$StopTime") }}, { "$out": "alltrips"} ]);` However it leads to the magic 1st of Jan 1970 being all over the place in both fields. – ffritz Jun 15 '17 at 14:35
  • Update: I know why, `ISODate` accepts my format of `"YYYY-MM-DD HH:MM"`, but `new Date` doesn't like it. – ffritz Jun 15 '17 at 14:51

0 Answers0