How to maintain history snapshot in MongoDB

Question

I have a huge object collection in mongoDB. These are basically products from a shopping website fetched from their APIs. Stored like:

{
    "id": "some_id"
    "baseInfo": {
        "identifiers": {
            "attributes": {
                "description": "some desc"
                "title": "some title"
                "price": {
                    "amount": "amount"
                    "currency": "currency"
                }
                "....": "..."
                ... snip ....
            }
        }
    }
}

This info keeps on changing and is updated by a python script hitting their API continuously.

What would be the easiest way to maintain what changes occurred to which object at what time. For example: in Nth run of the script, at time T1 the product with ID=I, changed it's price from P1 to P2. Similarly, N+1th run at time T2 the price was changed to P3. So basically what would be the easiest way to store the pricing history of the product with ID=I?

Please see my answer to this question: http://stackoverflow.com/a/30306918/299871 — iamjonesy, May 18 '15 at 15:23

score 0 · Answer 1 · answered Sep 24 '14 at 16:49

I would alter the python script to track the changes in a new collection. Use findAndModify to update the each product and retrieve the old version atomically, then push the old version into another collection of historical info, appending on a valid date (or date range) and possibly a version number. You could alternatively have the new collection store changes by computing the changes between old and new versions and storing that in the collection.

How to maintain history snapshot in MongoDB

1 Answers1