How to deserialize json which is sometimes a dictionary and sometimes an empty array?

Question

I need to deserialize JSON documents where a particular element is usually a dictionary, but when it contains no data it is an empty array. I have no control over the structure of the JSON. I don't need to serialize, only deserialize. I'm using Newtonsoft.Json 13.0.2, latest stable version.

In the following snippet, goodJson illustrates what these documents normally look like, and badJson shows what they look like when there are no details

using Newtonsoft.Json;

var goodJson = @"
{
    'id': 1,
    'details': {
        'apple': 1,
        'tomato': 4,
        'lettuce': 3,
        'onion': 2
    },
    'bar': { 'A': 5, 'B': 6 }
}";

var badJson = @"
{
    'id': 2,
    'details': [],
    'bar': { 'A': 7, 'B': 8 }
}";

var foo1 = JsonConvert.DeserializeObject<Foo>(goodJson);
Console.WriteLine($"goodJson: {foo1}");
var foo2 = JsonConvert.DeserializeObject<Foo>(badJson); // JsonSerializationException
Console.WriteLine($"badJson: {foo2}");

public class Foo
{
    public int Id { get; set; }
    public Bar Bar { get; set; } = new Bar();
    public Dictionary<string, int> Details { get; private set; } = new Dictionary<string, int>();

    public override string ToString()
    {
        return $"Id: {Id}, Bar: {Bar.A} {Bar.B}, Details count: {Details.Count}";
    }
}

public class Bar
{
    public int A { get; set; }
    public int B { get; set; }
}

This throws an exception at the commented line when deserializing badJson:

Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'System.Collections.Generic.Dictionary`2[System.String,System.Int32]' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly. To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array. Path 'details', line 4, position 16.

I understand the error - the deserializer is expecting an object, not an array. I can't change the type of the Details property of Foo to an array / list type, otherwise goodJson won't deserialize.

Having read Deserializing JSON when sometimes array and sometimes object and How to deserialize JSON data which sometimes is an empty array and sometimes a string value, which are similar to, but not quite the same as my scenario, it seems that overriding JsonConverter's ReadJson method is the way to go. So this is how I've gone about doing that:

using Newtonsoft.Json;
using System;

public class DictionaryOrArrayConverter<TKey, TValue> : JsonConverter
    where TKey : notnull
{
    public override bool CanConvert(Type objectType)
    {
        return true;
    }

    public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
    {
        switch (reader.TokenType)
        {
            case JsonToken.StartObject:
                var d1 = serializer.Deserialize(reader, objectType);
                Console.WriteLine(d1.GetType().FullName);
                var d2 = (Dictionary<TKey, TValue>)d1;
                Console.WriteLine($"In ReadJson: dictionary has {d2.Count} entries");
                return d2;
            case JsonToken.StartArray:
                reader.Read();
                if (reader.TokenType != JsonToken.EndArray)
                {
                    throw new JsonReaderException("Empty array expected");
                }

                return string.Empty;
        }

        throw new JsonReaderException("Expected object or empty array");
    }

    public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
    {
        serializer.Serialize(writer, value);
    }
}

And I decorate the Details property of Foo with my converter:

[JsonConverter(typeof(DictionaryOrArrayConverter<string, int>))]
public Dictionary<string, int> Details { get; private set; } = new Dictionary<string, int>();

Now the program runs without error, but the Details property of goodJson is deserializing as an empty dictionary. The console output is as follows:

System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]

In ReadJson: dictionary has 4 entries

goodJson: Id: 1, Bar: 5 6, Details count: 0

badJson: Id: 2, Bar: 7 8, Details count: 0

So I can tell that within the ReadJson method the details property of goodJson is deserializing to an object with a runtime type of Dictionary<string, int> with 4 entries, however the object returned by JsonConvert.DeserializeObject has no entries in its Details property.

I guess there must be something wrong with the way I've implemented ReadJson, but I can't see what it is.

Answer 1

The Details property has a private setter. By default, Newtonsoft.Json/Json.Net won't try assigning the dictionary created and returned by your DictionaryOrArrayConverter<,>.ReadJson method due to the setter being private.

Choose one of these options to solve your problem:

1. Make the Details setter public.
2. Annotate the Details property with a [JsonPropery] attribute, which tells Newtonsoft.Json to use the setter regardless of its private accessibility.
3. Since the Details property is already intialized with a dictionary instance, that dictionary instance should be available as the parameter existingValue in the converter's ReadJson method. Therefore, if existingValue is not null, populate this existing dictionary instance from existingValue with the json data and return it instead of creating a new dictionary instance.

Answer 2

I usually use a json constructor in this case

public class Foo
{
    //... your properties and code

    [JsonConstructor]
    public Foo(JToken Details)
    {
        if (Details.Type != JTokenType.Array)
            this.Details = Details.ToObject<Dictionary<string, int>>();
    }
    public Foo() {}
}

output

goodJson: Id: 1, Bar: 5 6, Details count: 4
badJson: Id: 2, Bar: 7 8, Details count: 0