Updating documents from a MongoDB collection to another

Question

I'm trying to backup some data that i stored on a collection to another collection hosted on a single db. I came up with a very simple solution, which works but very slowly, so i was trying to find a faster way to do this. Here is what i tried:

def updater():
    COL1= DB1.WData.find({})
    for x in COL1:
        Values = {"$set": {"datetime": x['datetime'], "amount": x['amount']}}
        Query = {"datetime": x['datetime'], "price": x['price']}
        DB2['COL2'].update_one(Query, Values, upsert=True)

As i said, this works but very slowly; and since i have around 23k records i was trying to find a way to speed it up. Any advice is appreciated

What is the purpose? Only backup and recovery incase of failure? Or copy to another table? — Gibbs, Aug 16 '20 at 16:05
Both of them; right now it's a backup, but i'm planning to use that db later — JayK23, Aug 16 '20 at 18:42

score 1 · Accepted Answer · answered Aug 17 '20 at 03:15

As you said, you are looking for two different things.

You can use mongo dump and mongo restore to backup and restore. Refer this documentation

You should not be taking backup to another collection. Because when something goes wrong with the sever, backup helps you to restore the data. When the server crashes, you can restore the data with the dump which is taken using mongodump. mongorestore helps to restore the dump.

If you just want to copy all the docs to another collection, you can use copyTo or $out or mongoexport as mentioned here

You can cron it as you want to do this every 12 hours once.

score 1 · Answer 2 · answered Aug 17 '20 at 05:11

1

If you can use aggregation, check the $merge pipeline stage. Documentation: $merge

answered Aug 17 '20 at 05:11

raga

899
10
14

Updating documents from a MongoDB collection to another

2 Answers2