Programmatic Manual JSON Deserialization into C# Object

Question

We have an extremely large flat C# object (20000ish properties) and we are deserializing a JSON object into it using the Newtonsoft JSON Deserializer.

Currently it is taking upwards of 3 minutes due to reflection.

We are looking to implement a programmatic manual deserialization process in order to cut down on this time.

We have read a lot of resources on this, these give a general idea on how to do manual deserialization but do not explain on how to approach it programmatically.

e.g.

http://www.tomdupont.net/2016/01/how-to-optimize-jsonnet-serialization.html

How to improve JSON deserialization speed in .Net? (JSON.net or other?)

For example, we have one large purchase object with 20000 properties and no subclasses/object properties, literally 20000+ strings. The way they are currently mapped from JSON is that we have a JSON property on the property itself which corresponds to the property on the JSON itself.

Reflection is very slow using this current approach and we are looking for a push in the right direction on how to do this programmatically.

The question you linked to suggests using [JsonReader/JsonWriter](https://stackoverflow.com/a/26383831/22437); did you try that? — Dour High Arch, Jun 06 '18 at 23:55
Yes we have tried it, it produced very marginal results only saving around 5 seconds. The issue is more with reflection rather than the large object heap. — Phillip M, Jun 07 '18 at 00:09
If it's just a single object with a bunch of string properties, is it any quicker to deserialise it to a `Dictionary`? — DavidG, Jun 07 '18 at 00:55

score 0 · Accepted Answer · answered Jun 07 '18 at 00:34

I'm not sure what you're asking for. It sounds like you want to use the JsonReader/JsonWriter approach, but you don't want to manually write 20,000 JSON-to-property assignments (e.g. if (token.Value == "prop1") result.prop1 = token.Value). If that's the case, then you could generate source code that does what you want, and include the result in your project. There are many ways to do that. Here's one way:

using System;
using System.IO;
using System.Linq;
using System.Text;

namespace GetCustomAttribute
{
    class MainClass
    {
        public static void Main(string[] args)
        {
            var sb = new StringBuilder();

            sb.AppendLine(@"
namespace GetCustomAttribute
{
    public static class PurchaseaOrderParser
    {
        public static void Parse(string jsonString, PurchaseOrder purchaseOrder)
        {

            var reader = new JsonTextReader(new StringReader(jsonString));

            var currentProperty = string.Empty;

            while (reader.Read())
            {
                if (reader.Value != null)
                {
                    if (reader.TokenType == JsonToken.PropertyName)
                        currentProperty = reader.Value.ToString();");

            var props = typeof(PurchaseOrder).GetProperties().Where(prop => Attribute.IsDefined(prop, typeof(IdAttribute)));
            foreach (var prop in typeof(PurchaseOrder).GetProperties())
            {
                var attribute = prop.GetCustomAttributes(typeof(IdAttribute), false).SingleOrDefault() as IdAttribute;
                if (attribute != null)
                {
                    var s = $"if (reader.TokenType == JsonToken.String && currentProperty == \"{attribute.Id}\") purchaseOrder.{prop.Name} = reader.Value.ToString();";
                    sb.AppendLine(s);
                }
            }

            sb.Append("}}}}}}");

            File.WriteAllText("PurchaseOrderParser.cs", sb.ToString());
        }
    }

    class PurchaseOrder
    {
        [Id("id")]
        public string Id { get; set; }

        [Id("name")]
        public string Name { get; set; }
    }

    class IdAttribute : Attribute
    {
        public string Id { get; set; }

        public IdAttribute(string id) => Id = id;
    }
}

which produces:

namespace GetCustomAttribute
{
    public static class PurchaseaOrderParser
    {
        public static void Parse(string jsonString, PurchaseOrder purchaseOrder)
        {

            var reader = new JsonTextReader(new StringReader(jsonString));

            var currentProperty = string.Empty;

            while (reader.Read())
            {
                if (reader.Value != null)
                {
                    if (reader.TokenType == JsonToken.PropertyName)
                        currentProperty = reader.Value.ToString();
if (reader.TokenType == JsonToken.String && currentProperty == "id") purchaseOrder.Id = reader.Value.ToString();
if (reader.TokenType == JsonToken.String && currentProperty == "name") purchaseOrder.Name = reader.Value.ToString();
}}}}}}

.NET also includes capabilities for runtime code generation. I'm not familiar with them, but you might be able to do the above at runtime, which would ensure the parser doesn't get out-of-sync with the PurchaseOrder class.

Programmatic Manual JSON Deserialization into C# Object

1 Answers1