Why does JsonConvert treat default constructors and parameterized constructors differently?

Question

I was trying out Json.net's ability to serialize and deserialize dictionaries, and thanks to this post I was able to find a good solution to serializing dictionaries in a simple way.

It was working great, but in a certain circumstance it broke in a (to me) nonsensical way such that I couldn't help but spend the next three hours debugging.

Here is the problem. When serializing this class

public class ReferenceTesting
{
    public List<Scenario> scenarios = new List<Scenario>();
    private Dictionary<Scenario, float> _Dict = new Dictionary<Scenario, float>();
    [JsonProperty]
    public List<KeyValuePair<Scenario, float>> SerializedDict
    {
        get { return _Dict.ToList(); }
        set { _Dict = value.ToDictionary(x => x.Key, x => x.Value); }
    }
    public ReferenceTesting(int number = 0)
    {
        for (int i = 0; i < number; i++)
        {
            Scenario s1 = new Scenario();
            scenarios.Add(s1);
            _Dict.Add(s1, i);
        }
    }
    public override string ToString()
    {
        string s = "";
        for (int i = 0; i < scenarios.Count(); i++)
        {
            Scenario scenario = scenarios[i];
            s += $"scenario{i} \n";
        }
        foreach (KeyValuePair<Scenario, float> scenario in SerializedDict)
        {
            s += $"Key: {scenario.Key}, Value: {scenario.Value} \n";
        }
        return s;
    }
}

Everything works as expected, meaning when I instantiate

new Reference(3);

and then serialize and deserialize, I end up with an object with as expected 3 items in the list, and 3 items in the dictionary.

Output:

scenario0 
scenario1 
scenario2 
Key: Scenario, Value: 0 
Key: Scenario, Value: 1 
Key: Scenario, Value: 2 

However, by adding the default constructor

public ReferenceTesting() { }

the serialization works, writing out 3 items in list and dictionary, but deserialization does not work with the property. Meaning I end up with

scenario0 
scenario1 
scenario2 

as output.

The big surprise with this is that the two constructors do the exact same thing - which is nothing when number = 0 (which it is when Json.net creates it, I doublechecked). So this means that the serializer has to be doing something under the hood to treat the property differently if there is or is not a default constructor.

Answer 1

How does Json.NET deserialize an object differently when it has a parameterized constructor vs. when it has a default constructor?

Json.NET is a streaming deserializer. Whenever possible it deserializes as it streams through the JSON rather than preloading the complete JSON into an intermediate representation before final deserialization.

Thus, when deserializing a JSON object with a default constructor, it first constructs the corresponding .Net object. It then recursively populates the object's .Net members by streaming through the key/value pairs in the JSON until the end of the JSON object. For each pair encountered, it finds the corresponding .Net member. If the value is a primitive type, it deserializes the primitive and sets the value. But if the value is a complex type (JSON object or array) it constructs the child object if necessary, sets the value back in the parent, and then populates it recursively as it continues to stream.

However, when deserializing an object with a parameterized constructor, Json.NET cannot use this streaming algorithm and instead must first fully deserialize the JSON object to an intermediate table of deserialized .Net name/value pairs, matching each JSON value to its corresponding .Net constructor argument or property by name and then deserializing to the type declared in .Net. Only then can the object be constructed by passing the deserialized constructor parameters into the constructor, and setting the remainder as property values.

For details on this process, see

• JSON.net: how to deserialize without using the default constructor?
• How does JSON deserialization in C# work.

(There is a third algorithm for ISerializable objects which does not apply in your case.)

Why is my surrogate public List<KeyValuePair<Scenario, float>> SerializedDict property not deserialized correctly when deserializing via a default constructor?

The reason is explained in this answer to Why are all the collections in my POCO are null when deserializing some valid json with the .NET Newtonsoft.Json component, and arises about of the specifics of Json.NET's Populate() algorithm:

1. It calls the getter in the parent class to get the current value of the property being deserialized.

2. If null, and unless a custom constructor is being used, it allocates an instance of the property's returned type (using the JsonContract.DefaultCreator method for the type).

3. It calls the setter in the parent to set the allocated instance back into the parent.

4. It proceeds to populate the instance of the type.

5. It does not set the instance back a second time, after it has been populated.

Thus the setter for SerializedDict is not called after the list is populated.

But when the parent class has a parameterized constructor, the property value SerializedDict is fully deserialized before its parent is constructed, so the setter is called with a fully populated surrogate list.

How can I create a surrogate collection property that works in both scenarios?

You can use an array instead of a list. Since an array cannot be resized, it must be fully deserialized and populated before it can be set back in the parent object:

public class ReferenceTesting
{
    public KeyValuePair<Scenario, float> [] SerializedDict
    {
        get { return _Dict.ToArray(); }
        set { _Dict = value.ToDictionary(x => x.Key, x => x.Value);  }
    }
    // Remainder unchanged

You could make the array property be private if you want, by marking it with [JsonProperty].

By the way, your current deserialization creates duplicate Scenario objects in the scenarios and _Dict collections as shown by demo fiddle #1 here.

One way to fix this would be to serialize just _Dict (assuming that all scenarios are in the dictionary). Another would be to use PreserveReferencesHandling, e.g. by adding [JsonObject(IsReference = true)] to Scenario:

[JsonObject(IsReference = true)]
public class Scenario
{
    // Remainder unchanged
}

Notes:

• There is no standard for serialization of references in JSON. Json.NET's implementation may not match that of other serializers.

• PreserveReferencesHandling doesn't work for objects with parameterized constructors (see here for details), so make sure Scenario doesn't have one.

Demo fiddle #2 here showing everything working correctly with a default constructor, and #3 here with a parameterized constructor.