4

We have a large ReadOnlyDictionary that serves as a high-speed cache for a large number of data objects we need to have quick access to.

On a timer, we regularly load up a replacement cache, then when it's completely loaded, we replace the existing cache object with the newly created one so that operations continue without interruption.

Recently, we're running into some memory leak issues, and one of the things we're examining is this cache replacement.

A simplified write-up of what we are doing:

public class Optimizations
{
    private volatile ReadOnlyDictionary<string, Dictionary<string, object>> _cache;

    public object GetFromLocalCache(string dataSet, string key)
    {
        object obj;
        Dictionary<string, object> dict;
        if (_cache.TryGetValue(dataSet, out dict) && dict.TryGetValue(key, out obj))
        {
            return obj;
        }

        return null;
    }

    private void OnTimerEvent()
    {
        ReadOnlyDictionary<string, Dictionary<string, object>> newCache = ...
        _cache = newCache;
    }
}

I'm wondering if the data that was previously in the _cache is properly garbage collected, since we seem to be abandoning it for a new instance. Should we be doing something different to make sure the previous cache is properly destroyed, and the data is reclaimed by the OS?

Jeremy Holovacs
  • 22,480
  • 33
  • 117
  • 254
  • The missing part of the code (`...`) could potentially be interesting, in case the returned instance reference is hold somewhere else. – ken2k Sep 13 '16 at 15:38
  • @ken2k we've gone over that stuff pretty thoroughly, and we're pretty sure that's not an issue. Everything is loaded into new POCO's from either a db or a file. – Jeremy Holovacs Sep 13 '16 at 15:42
  • Have you tried a memory profiling tool? This should point out what part of your code still holds a reference to the old `_cache` instance once it's replaced by the new one. It's a bit hard to guess with the example you provided. I don't see an obvious issue here (you multi-thread without a proper `lock` though, but it shouldn't be relevant for your issue) – ken2k Sep 13 '16 at 15:48
  • @ken2k, side question, if there are no writes to that dictionary (it's readonly), why lock? – Evk Sep 13 '16 at 15:55
  • @Evk Imagine the following execution order: thread A executes `_cache.TryGetValue(dataSet, out dict)`, then thread B executes `_cache = newCache;`, now what when thread A executes `dict.TryGetValue(key, out obj)`? The value retrieved from the cache (`dict`) might not exists anymore in the new `_cache` instance, but will still be returned. It's not like it'll throw any exception, but you might encounter strange behaviors. – ken2k Sep 13 '16 at 15:59
  • @Evk Also see http://stackoverflow.com/a/17530556/870604 for the `volatile` stuff – ken2k Sep 13 '16 at 16:01
  • 1
    how large does your dictionaries get in terms of bytes. I ask because dictionaries use arrays internally. If your dictionaries get too large, they may be placed on the LOH (large object heap) by default. This may be contributing because the LOH isn't condensed on collection by default. Also any string literals dont get collected and are interned. With you using strings as your key, I would be careful about the number of literals used. – xtreampb Sep 13 '16 at 16:26
  • @ken2k yes, but all what can happen in both cases is that old cached item will be returned instead of new one, which seems to be not an issue (for cache). – Evk Sep 13 '16 at 16:28
  • @ken2k, unfortunately the memory profiler does not see anything in development, and screeches to a halt in production. It has been singularly unenlightening in identifying the source of the leak, so we're left with finding potential sources – Jeremy Holovacs Sep 13 '16 at 16:54
  • @xtreampb our data sets get large. Several million discrete objects, several hundred MB, often more. As for the string keys, not easy to get away from those, but maybe there's a good way to clean them up? – Jeremy Holovacs Sep 13 '16 at 16:59
  • Its highly unlikely you use string _literals_, and that are literals which are interned, so no reason to worry about that. You run your application on 64 bit machine? – Evk Sep 13 '16 at 18:29
  • @Evk, yes, it's all 64-bit. – Jeremy Holovacs Sep 13 '16 at 18:35
  • Then LOH should not be an issue either. I suppose without more info it will be hard to help you. – Evk Sep 13 '16 at 18:45
  • Well, I mainly wanted to know if what I gave an example of was a good way to do it, or if I was asking for trouble. It sounds like nobody sees a problem with what's been done, so I will continue to search for potential problems. It's easy to be something little when you're processing 30K transactions/ second... – Jeremy Holovacs Sep 13 '16 at 18:50
  • How did you establish that there is a memory leak? How frequently does the timer run? – Vikhram Sep 13 '16 at 19:03
  • @Vikhram timer runs every 5 minutes or so, but if it's still loading it exits without action. Memory leak usually occurs when deploying a new update, which made us think it was an orphaned timer process, but right now we're examining all possibilities. – Jeremy Holovacs Sep 13 '16 at 19:17

0 Answers0