Thread safe update of Cached Reference data

Question

Say I have several List properties. Something Like this:

List<CustomerTypes> CustomerTypes {get; set;}
List<FormatTypes> FormatTypes {get; set;}
List<WidgetTypes> WidgetTypes {get; set}
List<PriceList> PriceList {get; set;}

Because these values update very rarely, I am caching them in my WCF Service at startup. I then have a service operation that can be called to refresh them.

The service operation will query them all from the database something like this:

// Get the data from the database.
var customerTypes = dbContext.GetCustomerTypes();
var formatTypes = dbContext.GetFormatTypes();
var widgetTypes = dbContext.GetWidgetTypes ();
var priceList = dbContext.GetPriceList ();

// Update the references
CustomerTypes = customerTypes;
FormatTypes = formatTypes;
WidgetTypes = widgetTypes;
PriceList = priceList;

This results in very little time that these are not all in sync. However, they are not fully thread safe. (A call could access a new CustomerType and an old PriceList.)

How can I make it so that while I am updating the references, any use of these lists has to wait until all references have been updated?

Answer 1

First put all of those lists in to a single container class.

Class TypeLists
{
    List<CustomerTypes> CustomerTypes {get; set;}
    List<FormatTypes> FormatTypes {get; set;}
    List<WidgetTypes> WidgetTypes {get; set}
    List<PriceList> PriceList {get; set;}
}

Then replace the old property accesses with a function call.

private readonly object _typeListsLookupLock = new object();
private volatile TypeLists _typeLists;
private volatile DateTime _typeListAge;

public TypeLists GetTypeList()
{
    if(_typeLists == null || DateTime.UtcNow - _typeListAge > MaxCacheAge)
    {
        //The assignment of _typeLists is thread safe, this lock is only to 
        //prevent multiple concurrent database lookups. If you don't care that 
        //two threads could call GetNewTypeList() at the same time you can remove 
        //the lock and inner if check.
        lock(_typeListsLookupLock)
        {
            //Check to see if while we where waiting to enter the lock someone else 
            //updated the lists and making the call to the database unnecessary.
            if(_typeLists == null || DateTime.UtcNow - _typeListAge > MaxCacheAge)
            {
                _typeLists = GetNewTypeList();
                _typeListAge = DateTime.UtcNow;
            }
        }
    }
    return _typeLists;
}

private TypeLists GetNewTypeList()
{
    var container = new TypeLists()
    using(var dbContext = GetContext())
    {
        container.CustomerTypes = dbContext.GetCustomerTypes();
        container.FormatTypes = dbContext.GetFormatTypes();
        container.WidgetTypes = dbContext.GetFormatTypes();
        container.PriceList = dbContext.GetPriceList ();
    }
    return container;
}

The reason we change from a property to a function is you did

SomeFunction(myClass.TypeLists.PriceList, myClass.TypeLists.FormatTypes);

You could have TypeLists changed out from under you in a multi-threaded environment, however if you do

var typeLists = myClass.GetTypeLists();
SomeFunction(typeLists.PriceList, typeLists.FormatTypes);

that typeLists object is not mutated between threads so you do not need to worry about it's value changing out from under you, you could do var typeLists = myClass.TypeLists but making it a function makes it is more clear that you could potentially get different results between calls.

If you want to be fancy you can change GetTypeList() so it uses a MemoryCache to detect when it should expire the object and get a new one.

Answer 2

I thought it would be fun to put something together as an example. This answer is based on guidance from Marc Gravell's answer here.

The following class accepts a milliseconds value and provides an event to notify the caller that the refresh interval has been hit.

• It uses Environment.TickCount which is orders of magnitude faster than using DateTime objects.

• The double-checked lock prevents multiple threads from refreshing concurrently and benefits from the reduced overhead of avoiding the lock on every call.

• Refreshing the data on the ThreadPool using Task.Run() allows the caller to continue uninterrupted with the existing cached data.

  using System;
  using System.Threading.Tasks;
  
  namespace RefreshTest {
      public delegate void RefreshCallback();
  
      public class RefreshInterval {
          private readonly object _syncRoot = new Object();
  
          private readonly long _interval;
          private long _lastRefresh;
          private bool _updating;
  
          public event RefreshCallback RefreshData = () => { };
  
          public RefreshInterval(long interval) {
              _interval = interval;
          }
  
          public void Refresh() {
              if (Environment.TickCount - _lastRefresh < _interval || _updating) {
                  return;
              }
  
              lock (_syncRoot) {
                  if (Environment.TickCount - _lastRefresh < _interval || _updating) {
                      return;
                  }
  
                  _updating = true;
  
                  Task.Run(() => LoadData());
              }
          }
  
          private void LoadData() {
              try {
                  RefreshData();
  
                  _lastRefresh = Environment.TickCount;
              }
              catch (Exception e) {
                  //handle appropriately
              }
              finally {
                  _updating = false;
              }
          }
      }
  }
  

Interlocked provides a fast, atomic replacement of the cached data.

using System.Collections.Generic;

namespace RefreshTest {
    internal static class ContextCache {
        private static readonly RefreshInterval _refresher = new RefreshInterval(60000);
        private static List<int> _customerTypes = new List<int>();

        static ContextCache() {
            _refresher.RefreshData += RefreshData;
        }

        internal static List<int> CustomerTypes {
            get {
                _refresher.Refresh();

                return _customerTypes;
            }
        }

        private static void RefreshData() {
            List<int> customerTypes = new List<int>();  //dbContext.GetCustomerTypes();

            Interlocked.Exchange(ref _customerTypes, customerTypes);
        }
    }
}

Several million concurrent calls runs ~ 100ms (run your own tests though!):

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading.Tasks;

namespace RefreshTest {
    internal class Program {
        private static void Main(string[] args) {
            Stopwatch watch = new Stopwatch();
            watch.Start();

            List<Task> tasks = new List<Task>();

            for (int i = 0; i < Environment.ProcessorCount; i++) {
                Task task = Task.Run(() => Test());

                tasks.Add(task);
            }

            tasks.ForEach(x => x.Wait());

            Console.WriteLine("Elapsed Milliseconds: {0}", watch.ElapsedMilliseconds);
            Console.ReadKey();
        }

        private static void Test() {
            for (int i = 0; i < 1000000; i++) {
                var a = ContextCache.CustomerTypes;
            }
        }
    }
}

Hope that helps.

Answer 3

If you have a simple scenario maybe you can use a HACK.

Programatically edit your web.config (not important what you edit, you can invent a counter or go from 0 to 1 or back from 1 to 0 on some invented appSetting).

Look here for example.

This will allow all existing requests to finish and then it will restart your app domain inside IIS. At start of new app domain data from db will be reloaded into your lists.

Be WARNED that your'll also experience a delay on starting of new app domain (on 1st request, jitting IL again) and you will also loose your data in Session, Application, etc.

Advantage is that when running you don't have any performance hit because of locking.