2

I need to copy one collection to another server and change a value in every document. The collection is very large, therefore, to save time, instead of copying the collection and modifying the data, I was wondering if there could be a script that while it goes through the documents, it changes the value before it inserts the documents to another server. Modification is pretty much padding a field with 0s.

For example:

Source : server1:27017

db.students:

{"_id" : ObjectId("..."), "sId" : "1234", "grade" : 4 }
{"_id" : ObjectId("..."), "sId" : "4444", "grade" : 3 }
{"_id" : ObjectId("..."), "sId" : "5555", "grade" : 5 }

Destination: server2:27017

db.students:

{"_id" : ObjectId("..."), "sId" : "0001234", "grade" : 4 }
{"_id" : ObjectId("..."), "sId" : "0004444", "grade" : 3 }
{"_id" : ObjectId("..."), "sId" : "0005555", "grade" : 5 }
Firat Bakioglu
  • 77
  • 2
  • 9

1 Answers1

0

Your Source : server1:27017 and Destination: server2:27017 are two different servers and we need two different connections. It is clear that they are two different machines and hence you need to get the data from one server and modify it and save it in the other server. As you mentioned the modification is simpler, by just adding some leading zeroes, this can be done thru regex or a simple javascript function.

Alternate Approach:

If you don't want to keep both server connection at the same time, then you can go for below approach.

1) Export the collection using mongodump from Source : server1:27017

mongodump -d some_database -c some_collection

2) Restore the Dump using mongorestore to Destination: server2:27017

mongorestore -d some_other_db -c some_or_other_collection dump/some_collection.bson

3.1) Then a write a javascript function to retrieve and update all records with leading zeroes.

or try this

3.2) In aggregate use $project, $concat, $out

Note that we cannot update the same collection by referencing the same field(here prefixing id with zeroes), since this feature is not supported in MongoDB yet.

Please see SERVER-1765

MongoDB concatenate strings from two fields into a third field

Hope it Helps!

Community
  • 1
  • 1
Clement Amarnath
  • 5,301
  • 1
  • 21
  • 34
  • Thanks. Is there any way to modify the data before you insert ? I am aware of the mongo import and export. I am trying to save some time. – Firat Bakioglu Aug 17 '16 at 15:54
  • Since your collection is very large, do the process in the DB layer itself. Use mongodump, mongorestore in a temp collection, then use $project, $concat, $out together to store as the desired collection, finally remove the temp collection. – Clement Amarnath Aug 17 '16 at 16:13