0

Here is the situation: I have an application that is built around a central std::map instance. Inserts and updates are rare, but can happen. I now want to mirror this map to a database on disk on the fly (and keep it up to date). This work should be done by some thread in the background for performance reasons.

I am wondering what the best approach is. using std::shared_mutex I can have both the backup-thread and the main-thread read from the map simultaneously while ensuring the infrequent writes still work.

But how does the backup-thread iterate over the map? It cant use iterators as those are invalidated on insert.

[In addition, how to handle deletes best? Right now I have a protected vector<map_key> to_be_deleted but this requires full locking between the backup-thread and the main-thread...]

bernd feinman
  • 324
  • 2
  • 11
  • Who told you inserts invalidate the iterators? See this for the rules https://stackoverflow.com/questions/6438086/iterator-invalidation-rules – NathanOliver Nov 27 '17 at 23:48
  • Oh, that's nice indeed! But then we have the analogous problem with deletes, which do invalidate the iterator of the deleted element. (I didnt mention deletes specifically before bc I assumed inserts invalidated all iterators anyways.) – bernd feinman Nov 27 '17 at 23:54
  • If it is acceptable that different readers may see a different state of the map (they'd do anyway, e.g., when inserting/deleting an element) it may be reasonable to keep multiple versions of the map and give each reader a `std::shared_ptr` from a lock protected location: get the current map, iterate over it, let go of it. If it was updated in the meantime the next read would get the next state and the `shared_ptr` takes care of resource management. – Dietmar Kühl Nov 27 '17 at 23:55
  • @DietmarKühl good idea, but here the map is very large, so a full copy would not be possible (even if it happens rarely). – bernd feinman Nov 27 '17 at 23:58
  • I am thinking - the best way I see right now is: Insert/updates are done directly to the map, and the backup-thread iteratoes over it and does its thing. Deletes are handled by setting a flag in the map-values, and then the backup-thread gets a full lock on the map and deletes from both its backup-database and the map. What do you think? – bernd feinman Nov 28 '17 at 00:01
  • @berndfeinman: you may want to consider a [persistent datastructure](https://en.wikipedia.org/wiki/Persistent_data_structure) in this case. The basic idea is to shared the unmodified part between various versions. Essentially these data structures are more systematic approach to your flags. – Dietmar Kühl Nov 28 '17 at 00:11

0 Answers0