24

I am in the process of changing the schema for one of my MongoDB collections. (I had been storing dates as strings, and now my application stores them as ISODates; I need to go back and change all of the old records to use ISODates as well.) I think I know how to do this using an update, but since this operation will affect tens of thousands of records I'm hesitant to issue an operation that I'm not 100% sure will work. Is there any way to do a "dry run" of an update that will show me, for a small number of records, the original record and how it would be changed?

Edit: I ended up using the approach of adding a new field to each record, and then (after verifying that the data was right) renaming that field to match the original. It looked like this:

db.events.find({timestamp: {$type: 2}})
    .forEach( function (e) {
        e.newTimestamp = new ISODate(e.timestamp);
        db.events.save(e);
    } )

db.events.update({},
    {$rename: {'newTimestamp': 'timestamp'}},
    {multi: true})

By the way, that method for converting the string times to ISODates was what ended up working. (I got the idea from this SO answer.)

Community
  • 1
  • 1
bdesham
  • 15,430
  • 13
  • 79
  • 123

6 Answers6

8

Create a test environment with your database structure. Copy a handful of records to it. Problem solved. Not the solution you were looking for, I'm sure. But, I believe, this is the exact circumstances that a 'test environment' should be used for.

eidsonator
  • 1,319
  • 2
  • 11
  • 25
  • I thought of that. But if my first attempt turns out not to work, I'd have to clear the development environment and re-import the data--potentially many times--and that seemed like a hassle that I might be able to avoid. – bdesham May 17 '13 at 13:56
  • Yeah, I assumed that had been considered, just thought I'd throw the suggestion out there, anyway... I occasionally overlook the simple stuff sometimes, myself. – eidsonator May 17 '13 at 13:58
8

My advice would be to add the ISODate as a new field. Once confirmed that all looks good you could then unset the the string date.

James Wahlin
  • 2,811
  • 21
  • 22
2

In the several latest versions of MongoDB (at least starting with 4.2), you could do that using a transaction.

const { MongoClient } = require('mongodb')

async function main({ dryRun }) {
  const client = new MongoClient('mongodb://127.0.0.1:27017', {
    maxPoolSize: 1
  })

  const pool = await client.connect()
  const db = pool.db('someDB')

  const session = pool.startSession()

  session.startTransaction()

  try {
    const filter = { id: 'some-id' }
    const update = { $rename: { 'newTimestamp': 'timestamp' } }

    // This is the important bit
    const options = { session: session }

    await db.collection('someCollection').updateMany(
      filter,
      update,
      options // using session
    )

    const afterUpdate = db.collection('someCollection')
      .find(
        filter,
        options // using session
      )
      .toArray()

    console.debug('updated documents', afterUpdate)

    if (dryRun) {
      // This will roll back any changes made within the session
      await session.abortTransaction()
    } else {
      await session.commitTransaction()
    }
  } finally {
    await session.endSession()

    await pool.close()
  }
}

const _ = main({ dryRun: true })
NeverwinterMoon
  • 2,363
  • 21
  • 24
1

Select ID of particular records that you would like to monitor. place in the update {_id:{$in:[<your monitored id>]}}

Dewfy
  • 23,277
  • 13
  • 73
  • 121
  • That's one approach, although it still opens the possibility of destroying data if I screw up the update operation. – bdesham May 17 '13 at 14:55
  • 3
    @bdesham it is bad practice to experiment with live data without backup. It is thumb-finger-rule for any production manipulations - create rollback point first – Dewfy May 17 '13 at 16:53
1

Another option which depends of the amount of overhead it will cause you - You can consider writing a script, that performs the find operation, add printouts or run in debug while the save operation is commented out. Once you've gained confidence you can apply the save operation.

 var changesLog = [];
var errorsLog = [];
events.find({timestamp: {$type: 2}}, function (err, events) {
    if (err) {
        debugger;
        throw err;
    } else {
        for (var i = 0; i < events.length; i++) {
            console.log('events' + i +"/"+(candidates.length-1));
            var currentEvent = events[i];
            var shouldUpdateCandidateData = false;

            currentEvent.timestamp = new ISODate(currentEvent.timestamp);


            var change = currentEvent._id;
            changesLog.push(change);

            // // ** Dry Run **
            //     currentEvent.save(function (err) {
            //         if (err) {
            //             debugger;
            //             errorsLog.push(currentEvent._id + ", " + currentEvent.timeStamp + ', ' + err);
            //             throw err;
            //         }
            //     });
        }
        console.log('Done');
        console.log('Changes:');
        console.log(changesLog);
        console.log('Errors:');
        console.log(errorsLog);
        return;
    }
});
Kobi Barac
  • 111
  • 2
  • 4
1
db.collection.find({"_manager": { $exists: true, $ne: null }}).forEach(
    function(doc){
        doc['_managers']=[doc._manager];     // String --> List
        delete doc['_manager'];              // Remove "_managers" key-value pair
        printjson(doc);                      // Debug by output the doc result        
        //db.teams.save(doc);                // Save all the changes into doc data
    }
)

In my case the collection contain _manager and I would like to change it to _managers list. I have tested it in my local working as expected.

INDIAN2025
  • 191
  • 2
  • 6