Pause simultaneous REST calls until first one completes

Question

We have a REST API method similar to:

List<item> GetItems(int AccountID)
{
    var x = getFromCache(AccountID);
    if(x==null)
    {
        x = getFromDatabase(AccountID);
        addToCache(AccountID, x);
    }
    return x;
}

This is a fairly costly method with some complicated DB calls, and we have a common situation where hundreds of users with the same AccountId will make the call almost simultaneously (they are all being notified with a broadcast).

In the method, we cache the result set for 10 seconds since a near-time result is fine for everyone making the request within that window. However, since they all make the call at the same time (again, for a specific AccountID) the cache is never populated up front, so everyone ends up making the database call.

So my question is, within the method, how can I pause all incoming requests for a specific accountId and make them all wait for the first result set to complete, so that the rest of the calls can use the cached result set?

I've read a little about Monitor.Pulse and Monitor.Lock but the implementation for a per-accountId lock kind of escapes me. Any help would be tremendously appreciated.

is there a reason why many users are using the same AccountId is that a service account ID..? can you alter your stored procedure to use transactions or add some `With No Lock` command to the database side assuming you're using Sql Server..? — MethodMan, Oct 08 '15 at 23:20
I would consider a 2 level cache so if you have a second cache of "pending" that last 10 20 seconds then any calls that are currently in the pending cache will self block and wait a bit before trying the real db call. You may similarly want to cache "proven not exist" for a similar reason. I would avoid monitor and lock based on the value of the variables. I would reserve those locks and synchronizers for code and memory regardless of the value of the variables. — Sql Surfer, Oct 09 '15 at 00:16

score 2 · Accepted Answer · answered Oct 09 '15 at 01:03

You must lock on the same object for requests with the same AccountId but use different object for each individual AccountId. Here is example how to use Dictionary to keep track of locking object for individual AccountIds.

    Dictionary<int, Object> Locks = new Dictionary<int, object>();

    List<item> GetItems(int AccountID)
    {
        //Get different lock object for each AccountId
        Object LockForAccountId = GetLockObject(AccountID);

        //block subsequent threads until first thread fills the cache
        lock (LockForAccountId)
        {
            var x = getFromCache(AccountID);
            if (x == null)
            {
                x = getFromDatabase(AccountID);
                addToCache(AccountID, x);
            }
            return x;
        }
    }

    private Object GetLockObject(int AccountID)
    {
        Object LockForAccountId;

        //we must use global lock while accessing dictionary with locks to prevent multiple different lock objects to be created for the same AccountId
        lock (Locks)
        {
            if (!Locks.TryGetValue(AccountID, out LockForAccountId))
            {
                LockForAccountId = new Object();
                Locks[AccountID] = LockForAccountId;
            }
        }
        return LockForAccountId;
    }

score 0 · Answer 2 · answered Oct 09 '15 at 05:50

Have you thought about using Lazy<T> for this?

Try this code:

private object _gate = new object();
List<item> GetItems(int AccountID)
{
    lock (_gate)
    {
        var x = getFromCache(AccountID);
        if (x == null)
        {
            x = new Lazy<List<item>>(() => getFromDatabase(AccountID));
            addToCache(AccountID, x);
        }
        return x.Value;
    }
}

You would need to change getFromCache & addToCache to have the following signatures:

Lazy<List<item>> getFromCache(int AccountID)
void addToCache(int AccountID, Lazy<List<item>> x)

Pause simultaneous REST calls until first one completes

2 Answers2