Newtonsoft JSON.NET create a Type dictionary

Question

I have some Serialization code based on the Newtonsoft Json.NET package.
I serialize large amount of instances of few types, but JSON.NET add a tag e.g. "$type": "complex_serializer_tests.SerializerTests+Node, complex-serializer-tests, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null", to every element.

this adds a significant amount of size to the save-format, which i would like to eliminate.

I wanted to create a Type Dictionary, that will:
1. for every new type, assign an Id (integer) 2. and use it across the JSON something along the line of "$type":#105
while adding a type-id => type-name element.

I am sorry, this aint very specific,
but the problem is that I don't know how to address it and would love some guidance what topics should I read...

EDIT Clarification, I don't mind the $type property name, but it's content... instead of writing the assembly-full-qualified-name, i'd like to have an index that will represent it.

Thanks

Answer 1

You can define custom types using a custom Serialization Binder.

I.E.

public class MyBinder : ISerializationBinder
{
    public Dictionary<string,Type> Types { get; set; }

    public Type BindToType(string assemblyName, string typeName)
    {
        // probably want to add some error handling here
        return Types[typeName];
    }

    public void BindToName(Type serializedType, out string assemblyName, out string typeName)
    {
        assemblyName = null;
        // not very efficient, but could have a separate reverse dictionary
        typeName= Types.First(t => t.Value == serializedType).Value;
    }
}

var settings = new JsonSerializerSettings { SerializationBinder = new MyBinder { ... } };

Also, if it's adding type names where it can be inferred, you can specify when to add them in JsonSerializerSettings, though this may effect deserialization depending on the types you're deserializing to.

var settings = new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.None };
JsonConvert.SerializeObject(obj, settings);

Answer 2

I had similar requirements, here's how I did it:

• Create custom JsonConverter class
• Tell serializer to use you custom JsonConverter

JsonConverter Example

Please note that this is alpha code and you will have to change parts esp. GetAllItemTypes that initilaizes the type key to type map (known limitation: needs lock).

public class TypePropertyConverter : JsonConverter
{
    /// <summary>
    /// During write, we have to return CanConvert = false to be able to user FromObject internally w/o "self referencing loop" errors.
    /// </summary>
    private bool _isInWrite = false;

    public override bool CanWrite => !_isInWrite;

    private static Dictionary<string, Type> _allItemTypes;
    public static Dictionary<string, Type> AllItemTypes => _allItemTypes ?? (_allItemTypes = GetAllItemTypes());

    /// <summary>
    /// Read all types with JsonType or BsonDiscriminator attribute from current assembly.
    /// </summary>
    /// <returns></returns>
    public static Dictionary<string, Type> GetAllItemTypes()
    {
        var allTypesFromApiAndCore = typeof(TypePropertyConverter)
            .Assembly
            .GetTypes()
            .Concat(typeof(OrdersCoreRegistry)
                .Assembly
                .GetTypes());

        var dict = new Dictionary<string, Type>();
        foreach (var type in allTypesFromApiAndCore)
        {
            if (type.GetCustomAttributes(false).FirstOrDefault(a => a is JsonTypeAttribute) is JsonTypeAttribute attr)
            {
                dict.Add(attr.TypeName, type);
            }
            else if (type.GetCustomAttributes(false).FirstOrDefault(a => a is BsonDiscriminatorAttribute) is BsonDiscriminatorAttribute bda)
            {
                dict.Add(bda.Discriminator, type);
            }
        }
        return dict;
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        _isInWrite = true;

        try
        {
            var type = value.GetType();
            var typeKey = AllItemTypes.First(kv => kv.Value == type).Key;

            var jObj = JObject.FromObject(value, serializer);
            jObj.AddFirst(new JProperty("type", typeKey));
            jObj.WriteTo(writer);
        }
        finally
        {
            _isInWrite = false;
        }
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
        {
            return null;
        }

        // we need to read and remove the "type" property first
        var obj = JObject.Load(reader);
        var typeKey = obj["type"];
        if (typeKey == null)
        {
            throw new InvalidOperationException("Cannot deserialize object w/o 'type' property.");
        }

        obj.Remove("type");

        // create object
        if (!AllItemTypes.TryGetValue(typeKey.Value<string>(), out var type))
        {
            throw new InvalidOperationException($"No type registered for key '{typeKey}'. Annotate class with JsonType attribute.");
        }

        var contract = serializer.ContractResolver.ResolveContract(type);
        var value = contract.DefaultCreator();

        if (value == null)
        {
            throw new JsonSerializationException("No object created.");
        }

        using (var subReader = obj.CreateReader())
        {
            serializer.Populate(subReader, value);
        }

        return value;
    }

    public override bool CanConvert(Type objectType)
    {
        return AllItemTypes.Any(t => t.Value == objectType);
    }
}

It's looking for a custom attribute "JsonType" and will use its Name properties value as key. If no JsonType is found, it will look for BsonDiscriminator attribute (from mongodb) as a fallback. You will havt to adjust this part.

Tell Serializer About You JsonConverter

There are multiple ways to do this. I'm using attributes like so:

Use converter for items of a list:

    [JsonProperty(ItemConverterType = typeof(TypePropertyConverter))]
    public List<PipelineTrigger> Triggers { get; set; }

See https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_Serialization_JsonProperty.htm for details.

Or you could add JsonConverter attribute to you base class: https://www.newtonsoft.com/json/help/html/JsonConverterAttributeClass.htm