Concurrently Access and Write to a Synchronized Dictionary Efficiently in C#

Question

I'm looking for the most efficient way to store key value pairs in a static Synchronized Dictionary (.NET 3.5, so not ConcurrentDictionary) while being able to access them at the same time.

Dictionary.Add(key, value);

if (Dictionary.Count >= 200)
{
       foreach (KeyValuePair<string, Info> pair in Dictionary)
       {
              Info entry = pair.Value;
              StoreInDatabase(entry);
       }

       Dictionary.Clear();
}

This is where the problem lies. If one user is adding to the dictionary while another is accessing and storing to the database it breaks.

lock (Dictionary)
{
    //Same Code Above
}

I put a lock in, and it seems to work fine, but I'm wondering if there is a more efficient way of doing this. It's not as efficient as I'd like it to be. Any suggestions would be much appreciated!

Note: I have to use the StoreInDatabase method to store the values.

REVISED CODE:

private static SynchronizedDictionary<string, Info> Dictionary = new SynchronizedDictionary<string, Info>();

...

Dictionary.Add(key, value);

if (Dictionary.Count >= 200)
{
       SynchronizedDictionary<string, Info> temporaryDictionary = new SynchronizedDictionary<string, Info>();
       lock (Dictionary)
       {
            temporaryDictionary = Dictionary;
            Dictionary.Clear();
       }

       lock(temporaryDictionary)
       {
            foreach (KeyValuePair<string, Info> pair in temporaryDictionary)
            {
                  Info entry = pair.Value;
                  StoreInDatabase(entry);
            }
       }
}

This greatly improved performance. Thanks flq!

IF (and only IF) you (may) have many readers and few concurrent writers then ReadWriterLockSlim is your friend. It'll allow to make multiple reads together, locking only for writing. — Adriano Repetti, May 30 '12 at 13:59
Is there a compelling reason you're using a Dictionary as a data structure? If you used something like a Queue you can efficiently read and write at the same time much more easily. — Servy, May 30 '12 at 14:02
There are lots of details that play into what kind of approach would be best. How many of the CRUD operations does the dictionary need to support? How many items can you expect to need space for? What's the ratio of reading to writing that you expect? — Sean U, May 30 '12 at 14:55
The ratio is 1:1. I'm writing until I reach 200 and then I read those values into the database (all 200) and clear the dictionary. — Brad Germain, May 30 '12 at 15:59
@Servy We used a dictionary for a fast way to check for duplicate keys. The code has been stripped down for simplicity. — Brad Germain, May 30 '12 at 17:24

score 3 · Accepted Answer · answered May 30 '12 at 13:59

You will have a lock around the DB operations which in comparison to in-memory activities take ages.

Within a lock, you should copy the values you want to store in the DB and clear the dictionary. Then you can release the dictionary and some other thread writes the stuff to DB.

Also it may make sense to use some private locking object in order to minimize the potential of deadlocks.

score 1 · Answer 2 · answered May 30 '12 at 14:39

I have implemented a thread-safe dictionary using Interlocked, which is the most lightweight synchronization mechanism available.

You can find the source code here. It's written for Fasterflect, a library that helps make reflection tasks easier and faster. The code uses an #ifdefine to conditionally enable the custom dictionary for .NET 3.5 as our benchmarks shows the .NET 4.0 ConcurrentDictionary to be even faster.

As flq points out, accessing the database while holding a lock is a really bad idea. Like seriously, not something you would ever want to do. Find a better solution, such as copying data you need to store to a temporary data structure.

Concurrently Access and Write to a Synchronized Dictionary Efficiently in C#

2 Answers2