How to Expire Many Items From a .NET MemoryCache

Question

What is the recommended way to remove a large number of items from a MemoryCache instance?

Based on the discussion around this question it seems that the preferred approach is to use a single cache for the whole application and use namespaces for keys to allow multiple logical types of items to be cached in the same instance.

However, using a single cache instance leaves the problem of expiring (removing) a large number of items from the cache. Particularly in the case where all items of a certain logical type must be expired.

At the moment the only solution I found was based on the answer to this question but it's really not very good from a performance stand-point since you would have to enumerate through all keys in the cache, and test the namespace, which could be quite time-consuming!

The only work-around I came up with at the moment is to create a thin wrapper for all the objects in the cache with a version number and whenever an object is accessed, discard it if the cached version doesn't match the current version. So whenever I need to clear all items of a certain type, I would bump up the current version number rendering all cached items invalid.

The work-around above seems pretty solid. But I can't help but wonder if there isn't a more straight-forward way to accomplish the same?

This is my current implementation:

private class MemCacheWrapper<TItemType> 
              where TItemType : class
{            
  private int _version;
  private Guid _guid;
  private System.Runtime.Caching.ObjectCache _cache;

  private class ThinWrapper
  {
     public ThinWrapper(TItemType item, int version)
     {
        Item = item;
        Version = version;
     }

     public TItemType Item { get; set; }
     public int Version { get; set; }
  }

  public MemCacheWrapper()
  {
      _cache = System.Runtime.Caching.MemoryCache.Default;
      _version = 0;
      _guid = Guid.NewGuid();
  }

  public TItemType Get(int index)
  {                
     string key = string.Format("{0}_{1}", _guid, index);

     var lvi = _cache.Get(key) as ThinWrapper;

     if (lvi == null || lvi.Version != _version)
     {
         return null;
     }

     return lvi.Item;
  }

  public void Put(int index, TItemType item)
  {                
     string key = string.Format("{0}_{1}", _guid, index);

     var cip = new System.Runtime.Caching.CacheItemPolicy();
     cip.SlidingExpiration.Add(TimeSpan.FromSeconds(30));

     _cache.Set(key, new ThinWrapper(item, _version), cip);
  }

  public void Clear()
  {
     _version++;                
  }
}

Answer 1

My recommended way to remove a large number of items from a MemoryCache instance is to use ChangeMonitor, and especially CacheEntryChangeMonitor.

Provides a base class that represents a ChangeMonitor type that can be implemented in order to monitor changes to cache entries.

So, it allows us to handle dependencies between cache items.

A vey basic example is

    var cache = MemoryCache.Default;
    cache.Add("mycachebreakerkey", "mycachebreakerkey", DateTime.Now.AddSeconds(15));

    CacheItemPolicy policy = new CacheItemPolicy();
    policy.ChangeMonitors.Add(cache.CreateCacheEntryChangeMonitor(new string[] { "mycachebreakerkey" }));
    // just to debug removal
    policy.RemovedCallback = args => { Debug.WriteLine(args.CacheItem.Key + "-->" + args.RemovedReason); };
    cache.Add("cacheKey", "cacheKey", policy);

    // after 15 seconds mycachebreakerkey will expire
    // dependent item "cacheKey" will also be removed

As for most of the things, you can also create a custom cache implementation or a derived change monitor type.

Not tested, but the CreateCacheEntryChangeMonitor suggests that you can create dependencies between MemoryCache.

Edit

ChangeMonitor is the .net way to invalidate content in the runtime cache. Invalidate means here = remove from the cache. It's used by SqlDependency or by a few asp.net components to monitor file change. So, I suppose this solution is scalable.

Here is a very simple benchmark, run on my laptop.

        const int NbItems = 300000;

        var watcher = Stopwatch.StartNew();
        var cache = MemoryCache.Default;

        var breakerticks = 0L;
        var allticks = new List<long>();

        cache.Add("mycachebreakerkey", "mycachebreakerkey", new CacheItemPolicy() { RemovedCallback = args => { breakerticks = watcher.ElapsedTicks; } });

        foreach (var i in Enumerable.Range(1, NbItems))
        {
            CacheItemPolicy policy = new CacheItemPolicy();
            if (i % 4 == 0)
                policy.ChangeMonitors.Add(cache.CreateCacheEntryChangeMonitor(new string[] { "mycachebreakerkeyone" }));
            policy.RemovedCallback = args => { allticks.Add(watcher.ElapsedTicks); };// just to debug removal
            cache.Add("cacheKey" + i.ToString(), "cacheKey", policy);
        }

        cache.Remove("mycachebreakerkey");
        Trace.WriteLine("Breaker removal=>" + TimeSpan.FromTicks(breakerticks).TotalMilliseconds);
        Trace.WriteLine("Start removal=>" + TimeSpan.FromTicks(allticks.Min()).TotalMilliseconds);
        Trace.WriteLine("End removal=>" + TimeSpan.FromTicks(allticks.Max()).TotalMilliseconds);
        Trace.WriteLine(cache.GetCount());

        // Trace
        // Breaker removal: 225,8062 ms
        // Start removal: 0,251 ms
        // End removal: 225,7688 ms
        // 225000 items

So, it takes 225 ms to remove 25% of my 300 000 items (again on my laptop, 3 year's old). Do you really need something faster ? Note, that the parent is remove at the end. Advantage of this solution :

• invalidated items are removed from the cache
• you are close to the cache (less callstack, less cast, less indirection)
• the remove callback allow you to auto-reload cache item if needed
• if the cachebreaker expire, then the callback is on another thread which will not impact asp.net requests.

I find your implementation pertinent and will keep it in mind for later. Your choice should be based upon your scenario : number of items, size of cache item, hit ratio, number of dependencies, ... also keeping too much data is the cache is generally slow and can increase probability of eviction.

Answer 2

Check out this post, and specifically, the answer that Thomas F. Abraham posted. It has a solution that enables you to clear the entire cache or a named subset.

The key thing here is:

// Cache objects are obligated to remove entry upon change notification.
base.OnChanged(null);

I've implemented this myself, and everything seems to work just fine.

Answer 3

If you use the "MemoryCache" implementation from "Microsoft.Extensions.Caching.Abstractions", which is targeted for .NET Standard, you can expire cache entries using CancellationTokens.

When creating a cache entry you can associate a CancellationToken with it.

For instance, you could create a CancellationToken "A" and associate it with a group of entries and a CancellationToken "B" to another group of entries. When cancelling CancellationToken "A", all entries associated with it are automatically expired.

You can run the sample code below to get a feeling about how this works.

using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Primitives;
using System;
using System.Threading;
using System.Threading.Tasks;

namespace Sample
{
    public class Program
    {
        public static async Task Main(string[] args)
        {
            var cache = new MemoryCache(new MemoryCacheOptions());
            var evenAgeCts = new CancellationTokenSource();
            var oddAgeCts = new CancellationTokenSource();

            var students = new[]
            {
                new Student() { Name = "James", Age = 22 },
                new Student() { Name = "John", Age = 24 },
                new Student() { Name = "Robert", Age = 19 },
                new Student() { Name = "Mary", Age = 20 },
                new Student() { Name = "Patricia", Age = 39 },
                new Student() { Name = "Jennifer", Age = 19 },
            };


            Console.WriteLine($"Total cache entries: {cache.Count}");

            foreach (var student in students)
            {
                AddToCache(student, student.Name, cache, student.Age % 2 == 0 ? evenAgeCts.Token : oddAgeCts.Token);
            }

            Console.WriteLine($"Total cache entries (after adding students): {cache.Count}");

            evenAgeCts.Cancel();
            Console.WriteLine($"Even aged students cancellation token was cancelled!");
            Thread.Sleep(250);

            Console.WriteLine($"Total cache entries (after deleting Student): {cache.Count}");

            oddAgeCts.Cancel();
            Console.WriteLine($"Odd aged students cancellation token was cancelled!");
            Thread.Sleep(250);

            Console.WriteLine($"Total cache entries (after deleting Bar): {cache.Count}");
        }

        private static void AddToCache<TEntry>(TEntry entry, string key, IMemoryCache cache, CancellationToken ct)
        {
            cache.GetOrCreate($"{entry.GetType().Name}\t{key}", e =>
            {
                e.RegisterPostEvictionCallback(PostEvictionCallback);
                e.AddExpirationToken(new CancellationChangeToken(ct));

                return entry;
            });
        }

        private static void PostEvictionCallback(object key, object value, EvictionReason reason, object state)
        {
            var student = (Student)value;

            Console.WriteLine($"Cache invalidated because of {reason} - {student.Name} : {student.Age}");
        }
    }

    public class Student
    {
        public string Name { get; set; }

        public int Age { get; set; }
    }
}

In the example I used the extension method "IMemoryCache.GetOrCreate" just for simplicity. You can easily achieve the same goal using method "IMemoryCache.CreateEntry".

Answer 4

The Cybermaxs's benchmark example is great. But it has an inaccuracy. At the line

policy.ChangeMonitors.Add(cache.CreateCacheEntryChangeMonitor(new string[] { "mycachebreakerkeyone" }));`

the cache key "mycachebreakerkeyone" shuld be "mycachebreakerkey". Because of this mistake 25% of items are deleted just after added to the cache. They do not wait the deletion of "parent" "mycachebreakerkey" to be deleted.