Migrating indexes with extra keys from Mongo 3.2 to Mongo 3.4

Question

I have the same problem as in this question :

MongoDB dump from 3.2, restore with 3.4, error index save = null

In my case, recreating indexes by hand is not an option, I need a script that automates this for migrating my production environment later.

What I have tried so far :

1/ running this in mongo shell on the new database:

for (var collection in ["_tempstore", "contracts", "images", "thumbs", "covers", "invoices"]) { 
  db.getCollection("cfs_gridfs." + collection + ".files").createIndex({filename: 1}); 
  db.getCollection("cfs_gridfs." + collection + ".chunks").createIndex({files_id: 1, n: 1}); 
}

which fails.

2/ Getting rid of the extraneous w key which is the root of the issue in my indexes on my old database by running :

db.system.indexes.update({w: {$exists: true}}, {$unset: {w: ""}})

which also fails.

What is the correct way to proceed?

Answer 1

I have written a script that i run against my dumped files to sanitize them.

First create those two files :

sanitize.sh

#!/usr/bin/env bash

DUMP_PATH=$1
for file in $( ls $DUMP_PATH | grep .*\.metadata\.json ); do
  node remove-extraneous-keys-from-indexes.js $DUMP_PATH/$file
done

remove-extraneous-keys-from-indexes.js

const fs = require("fs");
const {promisify} = require("util");

const fileName = process.argv[2];

(async () => {
  const text = await promisify(fs.readFile)(fileName, 'utf8')
  const json = JSON.parse(text)
  json.indexes = json.indexes.map(index => ({
    v: index.v,
    key: index.key,
    name: index.name,
    ns: index.ns
  }))
  await promisify(fs.writeFile)(fileName, JSON.stringify(json))
})()

then run

$ chmod u+x sanitize.sh
$ ./sanitize.sh path/to/dump/folder

Then when I run mongorestore, everything is fine.

WARNING : this script assumes you have the latest version of node running. Check this by running node -v. It should be 8.6 or more.