ConcurrentDictionary with limit

Question

I'm writing some code that uses reflection. So I'm trying to cache the expensive processing of the reflection into a ConcurrentDictionary. However, I want to apply a restriction limit on the concurrent dictionary to prevent storing old and unused cached values.

I did some research on my end to see how to limit the size of the ConcurrentDictionary. I found some interesting answers, however I don't know if the answers suits my requirements and will perform well.

I found in Dapper source code that they did some custom code to handle the limit of the ConcurrentDictionary. They have a collection limit constant that they use with Interlocked to be able to the handle the concurrency of the dictionary.

On the other hand, I found an answer on SO, that uses a normal Dictionary and then applies on it a ReaderWriterLockSlim to handle the concurrency. I don't know if it's the same implementation in .Net source code.

Should I go with dapper implementation or the SO answer implementation?

hey now, don't go thinking that "dapper" is perfect code; that was just the first thing that worked well enough, and we haven't yet had to go back and refactor it - it shouldn't be treated as an example of... well, anything, really; picking either example here - or a third option - is down to context (which we don't have) and opinion (which we don't want) — Marc Gravell, Mar 15 '18 at 15:54
For that use you can use the Cache classes that .Net already implements and some examples, I think it's the best way. — Oscar Vicente Perez, Mar 15 '18 at 15:57
@Marc Gravell: I know that it's a way of implementing the concurrent dictionary and that you can do it in a lot of ways. I thought of using a thread that will clean up the dictionary. I also thought of using a queue that will hold the last X records. I know that the answer is subjective and the implementation may vary.... — billybob, Mar 15 '18 at 16:23
Use a SemaphoreSlim to do the counting. Initialize it with the maximum you want, Wait() when add, Release() when TryAdd() returns false or remove succeeded. — Hans Passant, Mar 15 '18 at 16:30
Do you really have THAT much different reflection going on that caching it might become a memory problem? — Evk, Mar 15 '18 at 17:02
@HansPassant: it's exactly what is done in https://stackoverflow.com/questions/27403530/thread-safe-collection-with-upper-bound. It uses `ReaderWriterLockSlim` instead of a SemaphoreSlim — billybob, Mar 15 '18 at 17:44
That is very buggy, RWLS cannot ensure that multiple readers can't access the dictionary at the same time. — Hans Passant, Mar 15 '18 at 17:57
@HansPassant: So basically, I just use `SemaphoreSlim`, and I would be able to have a thread safe dictionary and can be accessible by multiple instance/readers at the same time? I'm also checking the implementation of `ConcurrentDictionary`. to see how they do it. https://referencesource.microsoft.com/#mscorlib/system/Collections/Concurrent/ConcurrentDictionary.cs,2e5ef5704344a309,references — billybob, Mar 15 '18 at 18:03
You will have to decide what to do when the limit is reached. Do you stop allowing user to add more entries? Or do you evict some old entries to accommodate the new one? This will lead to very different solutions. If you want to evict, none of the solutions you linked work. — Xiaoguo Ge, Mar 16 '18 at 03:06
Reflection already caches metadata internally upon first read. Why are you trying to build a cache on top of another cache? This sounds like an assignment problem to me where some brain dead teacher made using ConcurrentDictionary<,> a requirement. — Tanveer Badar, Apr 01 '18 at 08:59

score 1 · Answer 1 · answered Oct 08 '20 at 10:27

1

For performance, you should not use locking at all. Also, beware of hidden performance bottlenecks in the ConcurrentDictionary class, e.g. garbage is created when enumerating the collection (see issue here).

Use ThreadStatic

The only sane solution for a reflection cache, is to have a separate dictionary for each thread. Yes, this costs a bit of RAM, but the performance will be superior.

public static class TypeExtensions
{
    [ThreadStatic] private static Dictionary<Type, PropertyInfo> propertyInfoLookup;

    private static Dictionary<Type, PropertyInfo> MemberInfoLookup =>
        propertyInfoLookup ??= new Dictionary<Type, PropertyInfo>();

    // Sample API
    public static PropertyInfo GetPropertyInfo(this Type type) => MemberInfoLookup[type];
}

answered Oct 08 '20 at 10:27

l33t

18,692
16
103
180

I was not aware of the `ThreadStatic`! It is pretty cool. But how does it apply to the question? – billybob Oct 08 '20 at 16:01
1

Once you have an isolated, performant container you can easily loop through it continuously and remove cached values that are no longer wanted (e.g. via a timer). Certainly, I can improve my answer to include this. I just wanted to quickly protect OP's code from ending up with unwanted locking. :P – l33t Oct 08 '20 at 20:53
1

@billybob this answer defeats the purpose of reflection caching since each thread will start with an empty cache. your reflection code will run again and again for each request to your site (assuming this is for a web app). I don't recommend using ThreadStatic as the solution. – user3163495 Jul 17 '22 at 16:16

score 0 · Answer 2 · answered Mar 15 '18 at 16:02

0

The exact way to go about "Caching Information" varries a lot on Environment. Areas like WebDevelopment need totally different approaches, thanks to the massively paralell nature of the environemt and high level of seperation.

But the core thing to do caching of anything is WeakReference. Strong References prevent the collection by the GC. WeakReferences do not, but allow you to get strong references. It is the programmers way of saying "Do not keep it in memory just for the sake of this list. But if you have not collected it yet, give me a strong reference please.":

https://msdn.microsoft.com/en-us/library/system.weakreference.aspx

By it's nature, the GC can only collect (or tag weak references for that mater) when all other Threads are paused. So WeakReferences should not expose you to additional race conditions - it is either still there and you now have a strong reference, or it is not.

answered Mar 15 '18 at 16:02

Christopher

9,634
2
17
31

It's really important for me to to use `ConcurrentDictionary`. I will read about WeakReference to check if it might help me. – billybob Mar 15 '18 at 16:27
@billybob *why* is it really important to you to use `ConcurrentDictionary`? you'd actually be amazed how much you can do with `Hashtable` (yes, I really mean `Hashtable`, not `Dictionary<,>`) if both key and value are ref-type - it actually has an excellent threading model, despite having a weak type model – Marc Gravell Mar 15 '18 at 16:40
1

`WeakReference` can certainly be *part* of a cache implementation, but it isn't automatic that you **want** things to be collectable - sometimes, the value of the cache is stronger than simply GC requirements, so you need other mechanisms for eviction - TTL, LRU, etc – Marc Gravell Mar 15 '18 at 16:42
@MarcGravell: I think I remember some caching classes that had features like that. One even registers some unmanaged memory to do it's work in. I think I found a listing for this: https://learn.microsoft.com/en-us/dotnet/framework/performance/caching-in-net-framework-applications Apparently they added a whole namespace for this with 4.0. – Christopher Mar 15 '18 at 17:01
1

@MarcGravell: I don't see the utility of using a `Hashtable` and then re-implement the concurrency all by myself. I see that there's 2 approaches to the problem. Either, I take an existing implementation; `ConcurrentDictionary` in this case and add on it the limit restriction. Or, I re-code the concurrency over the Dictionary and apply the restriction limit over it. Also, I don't want to use a HashTable, since it's not generic, and I might use the 'limited dictionary' in other places. – billybob Mar 15 '18 at 17:55
@billybob: The only things Threadsave in the Concurrent classes are add and remove operations. Contains, then remove/add is still just as likely to have a update race condition because it is two distinct operations between wich no locking exists. You have to apply locking around this as usual. So, do not think the ConcurrentDictionary provides that much safety for you. It is a step up, but it can not possibly deal with everything. – Christopher Mar 15 '18 at 17:59
`WeakReference` is a nice little class for sure, but be aware that it consumes some additional memory and it might introduce indeterministic code (i.e. a dangling event handler is called until `GC`). For instance, a weak event handler occupies 10 times more memory than a strong event handler. – l33t Oct 08 '20 at 21:01

ConcurrentDictionary with limit

2 Answers2

Use ThreadStatic

Linked