1

I have two clusters of MongoDB which run on 4 servers (2 servers on each cluster). They are the same as collections schema. And now I want to migrate new data from one cluster to another which existing old data.

I need to migrate data from one cluster to another with large data (about 700GB).

I've tried using import/export of MongoDB Compass, DB tools such as mongodump/mongorestore, mongoexport/mongoimport. All of them work as expected for testing. But it is small data. With large data, they will be very slow to export from DB source and import to destination BD. And maybe they will crash while export/import large data. So, What is the best way I can do that? I've an idea to use replica sync mechanism for performance and safe reason.

  1. Use one mongo server of the source cluster as a Primary instance (server A), and one of the destination clusters as a Secondary instance (server B) and join them on the same replicaset. And the question is 700GB of new data on source server will sync to the destination server without removing existing data by replica sync mechanism or not?
  2. If step 1 works, after replica sync is complete, I will turn off replicaset between server A and server B and re-config the replica set for 2 servers on destination cluster with server B as Primary instance, and 2nd server of destination cluster as the Secondary instance. New data which be synced on server B will sync to this secondary instance.

That is my idea, and I don't know it will work on large data or not. Have any suggestions for me in my case? Thanks everyone for any suggestions.

Ali
  • 131
  • 1
  • 3
  • 12
AcidBurn
  • 73
  • 2
  • 15

1 Answers1

0

Option 1: ( small database size < 500GB )

mongodump/mongorestore -> if you have small database and you have other detabases that need to be preserved in the targeted server.

Option 2: ( any size )

Add the targeted server as SECONDARY/HIDDEN to the source , it will sync for some time , later just split it from the source and reconfigure with {force:true} as PRIMARY , then you can add more members in the new cluster for redundancy.

Option 3: ( big database size > 500GB )

Do file system snapshot , copy to the new server , reconfigure as new server.

R2D2
  • 9,410
  • 2
  • 12
  • 28
  • Thanks @R2D2 for quickly reply. As your suggestions, my case is Option 3. But I want to keep old data on destination cluster. It will be merged with new data from source cluster. If I use snapshot of source data and restore on destination by snapshot files, my old data be lossed or not ? – AcidBurn Jun 25 '22 at 09:28
  • 1
    previously in the mmap storage engine it was possible to merge partitions , but now in wiredTiger it is not , so if you want to preserve the old data it is best to use mongodump/mongorestore even it can take some more time ... – R2D2 Jun 25 '22 at 09:32
  • Maybe if your old database is small on targeted aeever you can mongodump/mongorestore after you replicate the new big database... – R2D2 Jun 25 '22 at 09:42
  • my mongo is using wiredTiger as storage engine. My old data is very large on destination cluster. That is why I migrate data from new cluster to old cluster (data on new cluster is smaller). So what do you think about my idea ? using replica sync mechanism to sync large data from one node on new cluster to one node on destination cluster. – AcidBurn Jun 25 '22 at 10:00