How can cached value be invalidated in single thread?

Question

I use the Http.Current.Cache to store various values retrieved from my database because my app is to data-intensive. When running my website on my new laptop with VS2017 install (on another laptop with VS2015 I never see this problem), I'm seeing a very strange issue where cached values seem to be randomly cleared- almost in a way to defies logic.

For instance, I have an if clause whose condition is that the cache item in question is not null. My code is definitely following the path through this if statement, but a couple of statements later the debugger shows that the cache item is in fact null- causing my app to fail.

public static SportSeason GetCurrentSportSeason(string sportCode)
{
  SportSeason sportSeason = null;
  string key = "Sheets_SportSeason_" + sportCode;

  int i = 0;
  if (BaseSheet.Settings.EnableCaching && BizObject.Cache[key] != null)
  {
    i = 1;
    sportSeason = (SportSeason)BizObject.Cache[key];
  }
  else
  {
    i = 2;
    sportSeason = GetSportSeasonFromSportSeasonDetails(SiteProvider.Sheets.GetCurrentSportSeason(sportCode));
    BaseSheet.CacheData(key, sportSeason);
  }
  if(sportSeason == null)
  {
    int j = i;
  }
  return sportSeason;
}

I can set a breakpoint in the final if and the variable i is set to 1, but the sportSeason object is NULL (as is the cache entry). How can this be when the only way the code could have entered the first if clause is if the cache item was not null?

Here is a screenshot showing some watch variables with a breakpoint in the final if.

This is tough to track down because it happens randomly throughout my business objects. Sometimes I have to refresh the page 3 or 4 times before I see this issues.

How can the cache be invalidated so quickly, and by what? The screenshot shows that there aren't many cached items, so I don't think I'm running out of memory. And no other processes are running which could be clobbering the cache.

EDIT: Through more debugging I've determined that only the cache key that is checked on Line 91 is cleared (set to null) when it is checked again at the breakpoint. All of the other cache records are still there.

EDIT2: I've isolated the problem down to this, although it still seems to defy logic:

    HttpContext.Current.Cache.Insert("ABC", "123", null, DateTime.Now.AddSeconds(600), TimeSpan.Zero);
    int i = 0;

I clear all my cache. When I step over the Cache.Insert statement, my cache count goes to 1. However, the cached item according to that key remains null (see watch window).

Also, if I execute one more statement (the int i = 0), the cache count goes back to zero.

Edit3: It's the AbsoluteExpiration parameter of the Insert() that is killing me. Somehow a clock is off.

If I set the AbsoluteExpiration to 5 hours into the future (300 minutes), it doesn't work. If I set it to 5 hours and one minute (301 minutes) into the future, everything works.

    HttpContext.Current.Cache.Insert("ABC", "123", null, DateTime.Now.AddMinutes(301), TimeSpan.Zero);

The clock on my laptop is 100% accurate and you can see the Intellisense shows the correct time as well. Can the cache be based off of some other clock that is 5 hours off?

Edit4: Looks like someone else is off by 5 hours.

Answer 1

The fix is to always use DateTime.UtcNow instead of DateTime.Now when specifying the AbsoluteExporation parameter.

Do this:

HttpContext.Current.Cache.Insert("ABC", "123", null, DateTime.UtcNow.AddMinutes(1), System.Web.Caching.Cache.NoSlidingExpiration);

Not this:

HttpContext.Current.Cache.Insert("ABC", "123", null, DateTime.Now.AddMinutes(1), System.Web.Caching.Cache.NoSlidingExpiration);

When I do this, it correctly recognizes that the cache is valid and I can retrieve the value. As to why this is required on my Windows 10 Pro laptop but not on my Windows 10 Home, I have no idea.

Answer 2

You can add a callback function that can tell you why an item was removed:

https://msdn.microsoft.com/en-us/library/7kxdx246.aspx

You specify the callback when adding an item, and it will be fired when the item is removed. Put a breakpoint or logging output in there to print out the CacheItemRemovedReason argument.

If that doesn't help you, you might try the debugging steps laid out in the top answer to ASP.NET HttpContext Cache removes right after insertion. This may catch if the app is terminating.

Answer 3

Review your IIS application pool recycling options to rule out a recycle event. Because an application pool recycle, will clear your cache.

Per the App pool recycling settings docs:

You can specify that IIS recycle an application pool at set intervals (such as every 180 minutes), at a specific time each day, or after the application pool receives a certain number of requests. You can also configure the element to restart the application pool when the worker process virtual memory and physical memory usage reaches a specific threshold.

Even changing your config can recycle the app pool.

Ensure that no other apps are sharing your app pool and then turn on recycle logging and reproduce the issue. See if the cache disappearance is correlated with an app pool recycle in the log. If so, go from there to determine why the recycle is happening and then make a configuration change so app pool doesn't recycle during normal operating hours.