What's the best way to cache results of a json.net serialization in memory?

Question

Project is MVC WebAPI based.

We are passing permission context of a client to our API servers as a serialized JSON object in the claims headers of the request. This is not a huge object: 6 properties and one collection of enum-based key-value pairs (up to 6 items here)

Vast majority of the requests to API occur every minute (some more frequently) from the same set of clients. Probably 700-900 clients (and growing), each one sending the same claims over and over, every minute.

For every request, various components of the code deserialize this object probably 5-6 times. This deserialization causes significant CPU drain on the servers.

What would be the best way to cache these deserializations in memory? Would a static Dictionary object with keys being serialized JSON strings, work well or would searching thru it be too slow, as these strings would be decently large in size?

EDIT: Every Action of every controller gets filtered thru this attribute to ensure that calls have proper permissions

    public class AccountResolveAttribute : ActionFilterAttribute
{
    public override void OnActionExecuting(HttpActionContext context)
    {
        var controller = (ControllerBase) context.ControllerContext.Controller;
        var identity = (ClaimsIdentity) controller.User.Identity;

        var users = identity.Claims
            .Where(c => c.Type == ClaimTypes.UserData.ToString())
            .Select(c => JsonConvert.DeserializeObject<UserInformation>(c.Value))
            .ToList();

        var accountId = controller.ReadAccountIdFromHeader();

        if (users.All(u => u.AccountId != accountId))
        {
            throw new ApplicationException(string.Format("You have no rights for viewing of information on an account Id={0}", accountId));
        }
    }
}

There are also calls in the base controller that interrogate the claims, but AccountResolve could probably cache the result of the first deserialization into the controller so that those calls do not try to deserialize again. However, the claims are the same over and over again, and I'm just trying to find a way to optimize to not deserialize again and again the same string. I've tried caching the serialization string as a key and result object into memory in a global static ConcurrentDictionary, but it doesn't appear to have helped

You say various components of the code deserialize the object. Out of curiosity, how are the components getting to the serialized data in the first place? For example: are the getting it from the current thread's principal, is the principal being being passed as a parameter, etc? — Rafael Dowling Goodman, May 07 '16 at 21:06
Another question in addition to the one I just asked: are you using an IoC container, and if so, which one? — Rafael Dowling Goodman, May 07 '16 at 21:09
Principal is validated against controller actions (it's an attribute in the base controllrr). It's also used in some places in base controller to get default contexts — Igorek, May 07 '16 at 21:26
How are the components you mention getting at the serialized data? — Rafael Dowling Goodman, May 07 '16 at 21:30
Thx for the edit. That helps. When you say caching the results using ConcurrentDictionary didn't help (which seems like it would) what led you to conclude that it didn't help? — Rafael Dowling Goodman, May 07 '16 at 22:27
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/111318/discussion-between-rafael-goodman-and-igorek). — Rafael Dowling Goodman, May 07 '16 at 23:06

score 1 · Answer 1 · edited May 23 '17 at 11:45

1

Since every GET returns different results, you'll likely need to implement your own caching, which isn't terribly hard. You can use MemoryCache or HttpRuntime.Cache to store whatever data you want. There's a simple example at the bottom of the documentation.

One cache exists for each process, so if you have IIS configured for more than one worker process, each process will hold its own cache.

But this way, you can hold whatever data you want in cache. Then retrieve it and manipulate it however you need to before returning data to the client.

You just need to implement some kind of locking to make sure the same cached item is not written to by multiple threads at the same time. See here for some ideas about that.

Old answer:

If each user sees the same data, then you can use Strathweb.CacheOutput.WebApi2, which is available in NuGet. It might fit your needs.

It will cache results based on the URL sent. So if data is returned for /api/getmydata, the next call to /api/getmydata will get data from the cache. You set the cache expiration.

You decorate your actions with the CacheOutputAttribute:

[CacheOutput(ServerTimeSpan = 100)]
public List<string> GetMyData() {
    ...
}

But if one action can return different data depending on who the user is, then this won't work so easily.

edited May 23 '17 at 11:45

Community

1
1

answered May 03 '16 at 15:43

Gabriel Luci

38,328
4
55
84

Unfortunately GET actions always return different results and most calls are POSTs anyway. Thanks! – Igorek May 03 '16 at 15:44
You will probably need to implement something yourself then, so you can cache only the data you want. I updated my answer. – Gabriel Luci May 03 '16 at 15:55
That's what I was asking about, as to how to best roll my own. MemoryCache also performs serialization, so that won't be useful. I think I need a good key lookup strategy – Igorek May 03 '16 at 15:58
According to [this answer](http://stackoverflow.com/a/2163065/1202807), [HttpRuntime.Cache](https://msdn.microsoft.com/en-us/library/system.web.httpruntime.cache(v=vs.110).aspx) does not serialize anything. So that might be an option instead. – Gabriel Luci May 03 '16 at 16:09
By the way, where did you read that MemoryCache performs serialization? – Gabriel Luci May 03 '16 at 16:13

Rafael Dowling Goodman · Accepted Answer · 2016-05-08T04:34:12.943

There seems to be two aspects to this question:

What the title is asking
Something is eating up CPU cycles; the assumption is that it's due to the deserialization of UserInformation instances

For 1., seems like a ConcurrentDictionary would fit the bill assuming that there really are a reasonably finite number UserInformation possibilities (you mention this in the question); otherwise not only would you keep taking the serialization cost, you'd essentially have something that looked like a memory leak.

If you can safely make the assumption, here's an example:

public static class ClaimsIdentityExtensions
{
    private static readonly ConcurrentDictionary<string, UserInformation> CachedUserInformations = new ConcurrentDictionary<string, UserInformation>();
    public static IEnumerable<UserInformation> GetUserInformationClaims(this ClaimsIdentity identity)
    {
        return identity
            .Claims
            .Where(c => c.Type == ClaimTypes.UserData)
            .Select(c => CachedUserInformations.GetOrAdd(
                c.Value,
                JsonConvert.DeserializeObject<UserInformation>));
    }
}

You had mentioned that you tried to use ConcerrentDictionary, but it didn't help. I would be shocked if the performance of deserializing an object beat out a lookup in a ConcurrentDictionary (again, making the aforementioned assumption), even if the keys are "long" strings. Without examples of the UserInformation class, it's hard to know with 100% certainty from our end...however, here's an example that shows that given a UserInformation with an AccountId property, the ConcurrentDictionary approach beats out the brute-force-deserialization approach by an order of magnitude:

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Security.Claims;
using Newtonsoft.Json;

namespace ConsoleApplication2
{
    public class UserInformation
    {
        public int AccountId { get; set; }
    }

    public static class ClaimsIdentityExtensions
    {
        private static readonly ConcurrentDictionary<string, UserInformation> CachedUserInformations = new ConcurrentDictionary<string, UserInformation>();
        public static IEnumerable<UserInformation> GetUserInformationClaims(this ClaimsIdentity identity, bool withConcurrentDictionary)
        {
            if (withConcurrentDictionary)
            {
                return identity
                    .Claims
                    .Where(c => c.Type == ClaimTypes.UserData)
                    .Select(c => CachedUserInformations.GetOrAdd(
                        c.Value,
                        JsonConvert.DeserializeObject<UserInformation>));
            }

            return identity
                .Claims
                .Where(c => c.Type == ClaimTypes.UserData)
                .Select(c => JsonConvert.DeserializeObject<UserInformation>(c.Value));
        }
    }

    class Program
    {
        static void Main()
        {
            var identity = new ClaimsIdentity(new[]
            {
                new Claim(ClaimTypes.UserData, "{AccountId: 1}"),
                new Claim(ClaimTypes.UserData, "{AccountId: 2}"),
                new Claim(ClaimTypes.UserData, "{AccountId: 3}"),
                new Claim(ClaimTypes.UserData, "{AccountId: 4}"),
                new Claim(ClaimTypes.UserData, "{AccountId: 5}"),
            });

            const int iterations = 1000000;
            var stopwatch = Stopwatch.StartNew();
            for (var i = 0; i < iterations; ++i)
            {
                identity.GetUserInformationClaims(withConcurrentDictionary: true).ToList();
            }
            Console.WriteLine($"With ConcurrentDictionary: {stopwatch.Elapsed}");

            stopwatch = Stopwatch.StartNew();
            for (var i = 0; i < iterations; ++i)
            {
                identity.GetUserInformationClaims(withConcurrentDictionary: false).ToList();
            }
            Console.WriteLine($"Without ConcurrentDictionary: {stopwatch.Elapsed}");
        }
    }
}

Output:

With ConcurrentDictionary: 00:00:00.8731377
Without ConcurrentDictionary: 00:00:05.5883120

One quick way to know if the deserialization of the UserInformation instances is the cause of the suspiciously high CPU cycles, try commenting out and stubbing out any validation against UserInformation and see if the cycles are still high.

What's the best way to cache results of a json.net serialization in memory?

2 Answers2