0

I've trying to build a web API throttling mechanism in ASP.NET Core that "penalizes" excess usage in an exponentially-growing denial window e.g. first 3 tries are allowed, 4th is blocked if within 60 seconds, 5th will blocked if within 120s, 6th blocked for 240s, etc.

The latest .Net 7's API rate-limiting is great, but rate-limiting isn't throttling and I couldn't find a way to augment it enough, either by partitioning or otherwise, as it would lack that sliding window feature.

I tried different options using LazyCache, directly with IMemoryCache, and my favourite, IDistributedCache - and looked at what others did, such as suggested by @JsAndDotNet here. However, all these derivatives suffer from the same security flaw when the cache is full (flaw for throttling purposes, that is): Adding a cache entry when the cache is full doesn't return any error (as discussed here). In fact, it continues as if that entry was added successfully to the cache. You can catch that asynchronically through a callback - but at that point, access has been already granted.

An attacker could simply flood the cache memory until its full, virtually bypassing the throttling mechanism on all subsequent requests. When the cache memory is full, compaction is triggered - which is another bad thing for throttling security, because it get rids of potentially relevant entries in the cache.

My question is how do I avoid that "cache full" vulnerability and catch it synchronically so I can block all transactions until memory is available again?

Mosh
  • 1
  • If the attacker can spoof the metadata that the cache key is made of, doesn't this mean that they already hacked the system? – tymtam Dec 05 '22 at 07:37
  • No spoofing is needed in the attack scenario I mentioned. I meant that an attacker that wants to circumvent the throttling can simply make enough API calls to fill the cache (if the sizelimit is set on 1024, for example, it will only takes 1024 calls) and once that happens, every subsequent call won't enter the cache, therefore bypass the throttling. To make it clearer: assume the system is open source - so the attacker knows to make 1024 calls. – Mosh Dec 05 '22 at 07:58
  • Is each request stored in the cache?! – tymtam Dec 05 '22 at 10:49
  • No. Cache key can be session (IP:port), user ID, API key, etc. So assume a new record entry for every IP/port attempted, for example. – Mosh Dec 05 '22 at 15:25
  • For these types of keys how would the attacker make 1024 different entries? – tymtam Dec 05 '22 at 21:25
  • Specifically with IP/ports, it actually quite trivial to iterate over thousands, or even tens of thousands source ports locally to create such an attack. That would create more than enough entries. If it was Username, that would be unlimited. – Mosh Dec 05 '22 at 23:24
  • What's the reason for storing the source port in the key? You control the usernames so you know how big the cache needs to be to handle all possibilities. – tymtam Dec 06 '22 at 01:04
  • BTW. You didn't mention https://github.com/stefanprodan/AspNetCoreRateLimit so you may look at as well (but ~by default it uses IMemoryCache). – tymtam Dec 06 '22 at 01:17
  • Apologies, tymtam, I think I unintentionally contributed to your confusion. I meant using either the IP/port OR the username as cache keys. Not both. Some throttling are done on the transport, others are on identity (but not both). For our discussion sake, assume one. IP/Port is easier to describe as the attack vector is simpler. Also when throttling on usernames you have to cater for inexistant usernames and offer the same behavior to achieve better security - but it doesn't matter for this purpose. – Mosh Dec 06 '22 at 01:54
  • As for the github link - thanks for that! That's exactly what I was referring to in my second paragraph about the new .net 7 rate-limiter. Unfortunately, it only rate-limits and doesn't throttle (AFAIK), and doesn't seem to have a different handling of "cache capacity reached" - so I can only deduce it suffers from the identical (lack of) error handling and compaction limitations as IMemoryCache. Again, I appreciate you looking that up though. – Mosh Dec 06 '22 at 02:01

0 Answers0