47

I have a dictionary with a custom .net Type as Its key.I am trying to serialize this dictionary to JSON using JSON.net, However its not able to Convert Keys to Proper Value during Serialization.

class ListBaseClass
{
    public String testA;
    public String testB;
}
-----
var details = new Dictionary<ListBaseClass, string>();
details.Add(new ListBaseClass { testA = "Hello", testB = "World" }, "Normal");
var results = Newtonsoft.Json.JsonConvert.SerializeObject(details);
var data = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<ListBaseClass, string>> results);

This Give me --> "{\"JSonSerialization.ListBaseClass\":\"Normal\"}"

However if I have my Custom type as value in Dictionary it Works well

  var details = new Dictionary<string, ListBaseClass>();
  details.Add("Normal", new ListBaseClass { testA = "Hello", testB = "World" });
  var results = Newtonsoft.Json.JsonConvert.SerializeObject(details);
  var data = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, ListBaseClass>>(results);

This Give me --> "{\"Normal\":{\"testA\":\"Hello\",\"testB\":\"World\"}}"

Can Someone Suggest If I am hitting some limitation of Json.net or I am doing Something Wrong?

Shashwat Gupta
  • 617
  • 2
  • 6
  • 6

7 Answers7

42

This is a pretty common problem people run into when they start using serializers. While it is a little puzzling why the limitation you discovered exists, there is a simple solution.

When I first wrote this answer a number of years back, Gordon Bean had replied with a workable solution to the problem using a type converter. You probably don't want to use that approach, though. It works, but it provides a serialized string for output. If you are using JSON, this will give you a less than ideal result, since you really want a JSON representation of an object and not a string representation.

for example, let's suppose that you have a data structure that associates unique grid points with strings:

class Point
{
    public int x { get; set; }
    public int y { get; set; }
}

public Dictionary<Point,string> Locations { get; set; };

Using the TypeConverter override, you will get a string representation of this object when you serialize it.

"Locations": {
  "4,3": "foo",
  "3,4": "bar"
},

But what we really want is:

"Locations": {
  { "x": 4, "y": 3 }: "foo",
  { "x": 3, "y": 4 }: "bar"
},

There are several problems with overriding the TypeConverter to serialize / deserialize the class.

First, this is not JSON, and you might have to write additional custom logic to deal with serialize and deserialize it elsewhere. (perhaps Javascript in your client layer, for example?)

Second, anywhere else that uses this object will now produce this string, where previously it serialized properly to an object. For example,

"GridCenterPoint": { "x": 0, "y": 0 },

now serializes to:

"GridCenterPoint": "0,0",

You can control the TypeConverter formatting a little, but you cannot get away from the fact it is rendered as a string and not an object.

This problem isn't a problem with the serializer, since Json.NET chews through complex objects without missing a beat, it is a problem with the way that dictionary keys are processed. If you try taking the example object, and serializing a List or even a Hashset, you notice that there isn't a problem producing proper JSON. This gives us a much simpler way to solve this problem.

Ideally, we would like to just tell Json.NET to serialize the key as whatever object type it is, and not force it to be a string. Since that doesn't seem to be an option, the other way is to give Json.NET something that it can work with: a List<KeyValuePair<T,K>>.

If you feed a list of KeyValuePairs into Json.NET's serializer, you get exactly what you expect. For example, here is a much simpler wrapper that you could implement:

    private Dictionary<Point, string> _Locations;
    public List<KeyValuePair<Point, string>> SerializedLocations
    {
        get { return _Locations.ToList(); }
        set { _Locations= value.ToDictionary(x => x.Key, x => x.Value); }
    }

Update from @bmw15 comment:
You can make the Dictionary prop public, add a [JsonIgnore] to it, and make the list of KeyValuePairs private, with a [JsonProperty] attribute

This trick works, because keys in a kvp aren't forced into string format. Why does the type converter generate a string format, you ask? It beats the hell out of me. The Dictionary object implements the IEnumerable<KeyValuePair<TKey, TValue>> interface, so there shouldn't be any problem in serializing it in the same fashion as the list of kvps, since that is essentially what a dictionary is. Someone (James Newton?) made a decision when writing the Newtonsoft dictionary serializer that complex keys were too messy to deal with. There are probably some corner cases I have not considered that make this a much more sticky problem.

I think that this is a much better solution because it produces actual JSON objects, is technically simpler, and doesn't produce any side effects resulting from replacing the serializer.

Roger Hill
  • 3,677
  • 1
  • 34
  • 38
  • 1
    I made a few tweaks to this solution and found it to be the fastest to implement and also correct in the sense @roger-hill mentions. My tweaks were to make the Dictionary prop public, add a [JsonIgnore] to it, and make the list of KeyValuePairs private, with a [JsonProperty] attribute. I used a private [JsonConstructor] that accepted the list of keyValuePairs, but I imagine it would work with just the [JsonProperty] attribute as well. – bmw15 May 26 '20 at 14:36
  • @bmw15 It looks like it does not work with just the JsonProperty. You need to have the constructor set the list of KeyValuePairs explicitly, or JsonConvert simply ignores the property, even with [JsonProperty]. I tried both your approach and Roger Hill's. – TheForgot3n1 May 29 '20 at 07:00
  • I tried a lots of thing but this is the best solution I found – Clément Andraud Sep 18 '22 at 17:17
34

The Serialization Guide states (see section: Dictionaries and Hashtables; thank you @Shashwat for the link):

When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding ToString() for the key type or by implementing a TypeConverter. A TypeConverter will also support converting a custom string back again when deserializing a dictionary.

I found a useful example for how to implement such a type converter on Microsoft's "how-to" page:

Essentially, I needed to extend System.ComponentModel.TypeConverter and override:

bool CanConvertFrom(ITypeDescriptorContext context, Type source);

object ConvertFrom(ITypeDescriptorContext context,
                   System.Globalization.CultureInfo culture, object value);

object ConvertTo(ITypeDescriptorContext context, 
                 System.Globalization.CultureInfo culture, 
                 object value, Type destinationType);

It was also necessary to add the attribute [TypeConverter(typeof(MyClassConverter))] to the MyClass class declaration.

With these in place, I was able to serialize and deserialize dictionaries automatically.

EvilTak
  • 7,091
  • 27
  • 36
Gordon Bean
  • 4,272
  • 1
  • 32
  • 47
  • This worked for us, to resolve a problem with complex Dictionary keys. Where a generic class was being used _twice_ with different concrete implementations in the class nesting, our custom JsonConverter did not work using the declarative attribute on the complex key class - we had to pass it as an argument to `JsonConvert.SerializeObject`, which worked but was less desirable. – mungflesh Apr 11 '16 at 08:34
  • I also just got this working for a complex Dictionary key type. Note for others - this is independant of the JSONTypeConverter stuff in the JSON.Net package. Also, there were no JSON settings which needed to be used for this. This answer really does contain exactly what you need to do. I found it useful to put breakpoints inside the overriden functions initially, to gain better understanding of when they were used & what they were asked to do. – StayOnTarget Feb 23 '17 at 15:01
  • Great answer, even though it did not work for me, since I am using windows phone 8.1 where System.ComponentModel.TypeConverter is not supported. – user3141326 Apr 19 '17 at 16:47
3

Another way to accomplish this is to use a custom ContractResolver and setting OverrideCreator.

public class DictionaryAsArrayResolver : DefaultContractResolver
{
    public override JsonContract CreateContract(Type objectType)
    {
        if (IsDictionary(objectType))
        {
            JsonArrayContract contract = base.CreateArrayContract(objectType);
            contract.OverrideCreator = (args) => CreateInstance(objectType);
            return contract;
        }

        return base.CreateContract(objectType);
    }

    internal static bool IsDictionary(Type objectType)
    {
        if (objectType.IsGenericType && objectType.GetGenericTypeDefinition() == typeof(IDictionary<,>))
        {
            return true;
        }

        if (objectType.GetInterface(typeof(IDictionary<,>).Name) != null)
        {
            return true;
        }

        return false;
    }

    private object CreateInstance(Type objectType)
    {
        Type dictionaryType = typeof(Dictionary<,>).MakeGenericType(objectType.GetGenericArguments());
        return Activator.CreateInstance(dictionaryType);
    }
}

Usage:

JsonSerializer jsonSerializer = new JsonSerializer();
jsonSerializer.ContractResolver = new DictionaryAsArrayResolver();
Tal Aloni
  • 1,429
  • 14
  • 14
  • An even better way is to map IDictionary<,> to Dictionary<,> (when calling CreateArrayContract), this would prevent Newtonsoft.Json from adding the Dictionary type information. this would also make setting the OverrideCreator not necessary. – Tal Aloni Sep 03 '20 at 17:49
2

After @roger-hill answer I came up with a lightweight solution to achieve the same result:

    [JsonArray]
    public class MyDictionary<K, V> : Dictionary<K, V>
    {
    }

In this way every MyDictionary object gets serialized as an array of Key/Value pair, behaving properly also with complex key type:

[{
    "Key": ...,
    "Value": ...
}, ...]
0

Inspired from gson enableComplexMapKeySerialization and how that looks\works:

public class DictionaryAsArrayJsonConverter : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var dictionary = (IDictionary)value;

        writer.WriteStartArray();

        var en = dictionary.GetEnumerator();
        while (en.MoveNext())
        {
            writer.WriteStartArray();
            serializer.Serialize(writer, en.Key);
            serializer.Serialize(writer, en.Value);
            writer.WriteEndArray();
        }
        
        writer.WriteEndArray();
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (!CanConvert(objectType))
            throw new Exception(string.Format("This converter is not for {0}.", objectType));

        Type keyType = null;
        Type valueType = null;
        IDictionary result;

        if (objectType.IsGenericType)
        {
            keyType = objectType.GetGenericArguments()[0];
            valueType = objectType.GetGenericArguments()[1];
            var dictionaryType = typeof(Dictionary<,>).MakeGenericType(keyType, valueType);
            result = (IDictionary)Activator.CreateInstance(dictionaryType);
        }
        else
        {
            result = (IDictionary)Activator.CreateInstance(objectType);
        }

        if (reader.TokenType == JsonToken.Null)
            return null;

        int depth = reader.Depth;
        while (reader.Read())
        {
            if (reader.TokenType == JsonToken.StartArray)
            {
            }
            else if (reader.TokenType == JsonToken.EndArray)
            {
                if (reader.Depth == depth)
                    return result;
            }
            else
            {
                object key = serializer.Deserialize(reader, keyType);
                reader.Read();
                object value = serializer.Deserialize(reader, valueType);
                result.Add(key, value);
            }
        }

        return result;
    }

    public override bool CanConvert(Type objectType)
    {
        return typeof(IDictionary).IsAssignableFrom(objectType);
    }

}

Possibly create same json as Tal Aloni code, but as a JsonConverter instead of contract. More flexible as it can be used on selected properties with JsonConverterAttribute or on everything with JsonSerializerSettings.Converters.Add(...)

osexpert
  • 485
  • 1
  • 6
  • 18
0

Building upon the insightful response by @roger-hill, I created the following JsonConverter that will convert an IDictionary object to a List of KeyValuePair objects.

github link

public class ListDictionaryConverter : JsonConverter
{
    private static (Type kvp, Type list, Type enumerable, Type[] args) GetTypes(Type objectType)
    {
        var args = objectType.GenericTypeArguments;
        var kvpType = typeof(KeyValuePair<,>).MakeGenericType(args);
        var listType = typeof(List<>).MakeGenericType(kvpType);
        var enumerableType = typeof(IEnumerable<>).MakeGenericType(kvpType);

        return (kvpType, listType, enumerableType, args);
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var (kvpType, listType, _, args) = GetTypes(value.GetType());
        
        var keys = ((IDictionary)value).Keys.GetEnumerator();
        var values = ((IDictionary)value).Values.GetEnumerator();
        var cl = listType.GetConstructor(Array.Empty<Type>());
        var ckvp = kvpType.GetConstructor(args);
        
        var list = (IList)cl!.Invoke(Array.Empty<object>());
        while (keys.MoveNext() && values.MoveNext())
        {
            list.Add(ckvp!.Invoke(new []{keys.Current, values.Current}));
        }
        
        serializer.Serialize(writer, list);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var (_, listType, enumerableType, args) = GetTypes(objectType);
        
        var list = ((IList)(serializer.Deserialize(reader, listType)));

        var ci = objectType.GetConstructor(new[] {enumerableType});
        if (ci == null)
        {
            ci = typeof(Dictionary<,>).MakeGenericType(args).GetConstructor(new[] {enumerableType});
        }
        
        var dict = (IDictionary) ci!.Invoke(new object[]{ list });

        return dict;
    }

    public override bool CanConvert(Type objectType)
    {
        if (!objectType.IsGenericType) return objectType.IsAssignableTo(typeof(IDictionary));
        
        var args = objectType.GenericTypeArguments;
        return args.Length == 2 && objectType.IsAssignableTo(typeof(IDictionary<,>).MakeGenericType(args));
    }
}

I've done some testing and this code works well in those tests... but I maybe missing a edge case or two.

Hypo Dwarf
  • 11
  • 3
-4

Everything is easier

var details = new Dictionary<string, ListBaseClass>();
details.Add("Normal", new ListBaseClass { testA = "Hello", testB = "World" });
var results = Newtonsoft.Json.JsonConvert.SerializeObject(details.ToList());
var data = 
Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<ListBaseClass, string>> results);

Sample

 class Program
{
    static void Main(string[] args)
    {
        var  testDictionary = new Dictionary<TestKey,TestValue>()
        {
            {
                new TestKey()
                {
                    TestKey1 = "1",
                    TestKey2 = "2",
                    TestKey5 = 5
                },
                new TestValue()
                {
                    TestValue1 = "Value",
                    TestValue5 = 96
                }
            }
        };

        var json = JsonConvert.SerializeObject(testDictionary);
        Console.WriteLine("=== Dictionary<TestKey,TestValue> ==");
        Console.WriteLine(json);
        // result: {"ConsoleApp2.TestKey":{"TestValue1":"Value","TestValue5":96}}


        json = JsonConvert.SerializeObject(testDictionary.ToList());
        Console.WriteLine("=== List<KeyValuePair<TestKey, TestValue>> ==");
        Console.WriteLine(json);
        // result: [{"Key":{"TestKey1":"1","TestKey2":"2","TestKey5":5},"Value":{"TestValue1":"Value","TestValue5":96}}]


        Console.ReadLine();

    }
}

class TestKey
{
    public string TestKey1 { get; set; }

    public string TestKey2 { get; set; }

    public int TestKey5 { get; set; }
}

class TestValue 
{
    public string TestValue1 { get; set; }

    public int TestValue5 { get; set; }
}