Using pickle, I am saving dictionaries every time a key value changes (or when a new key is added). I am assuming that pickle saves all the data including the keys and values that have not changed. Is there a way to selectively pickle data when key changes? Something more like what git does i.e. save only what has changed.
Asked
Active
Viewed 162 times
1
-
2Lots and lots. Are you looking for a database, maybe, like SQLite? What needs to be saved and how does it need to be used? – Ry- Apr 06 '20 at 11:25
-
1It sounds like you want a database. – juanpa.arrivillaga Apr 06 '20 at 11:26
-
Does this answer your question? [Pickle alternatives](https://stackoverflow.com/questions/9897345/pickle-alternatives) – Mathieu Apr 06 '20 at 11:29
-
How large are your dictionaries? – Stefan Pochmann Apr 06 '20 at 11:30
-
It can be a few GB. It would have been perfect if pickle checked if part of data has changed. Thanks for the alternatives. I will check them. – nivesh Apr 06 '20 at 14:19
1 Answers
0
Sounds to me as if pickle no longer fits your requirements. Have a look at the Pickle Alternatives as suggested by Amin.
Another possibilty would be a sqlite-based key-value store: Use SQLite as a key:value store
Lydia van Dyke
- 2,466
- 3
- 13
- 25