11

I came across the below question which is mostly identical to the issue I am having:

JSON.NET cannot handle simple array deserialization?

However, my situation is slightly different. If I modify the Test class from that question to have an array property of the same type, I get the same deserialization error.

class Test
{
    public Test[] Tests;
}

var settings = new JsonSerializerSettings
{
    PreserveReferencesHandling = PreserveReferencesHandling.All
};

var o = new Test { Tests = new[] { new Test(), new Test() } };
//var o = new Test(); //this works if I leave the Tests array property null
var arr = new[] { o, o };
var ser = JsonConvert.SerializeObject(arr, settings);

arr = ((JArray)JsonConvert.DeserializeObject(ser, settings)).ToObject<Test[]>();

I bet I am missing an important attribute on the Tests property.

Community
  • 1
  • 1
oscilatingcretin
  • 10,457
  • 39
  • 119
  • 206

3 Answers3

11

Json.NET simply hasn't implemented preserving of references for read-only collections and arrays. This is explicitly stated in the exception message:

Newtonsoft.Json.JsonSerializationException: Cannot preserve reference to array or readonly list, or list created from a non-default constructor: Question41293407.Test[].

The reason that Newtonsoft has not implemented this is that their reference tracking functionality is intended to be capable of preserving recursive self references. Thus the object being deserialized must be allocated before reading its contents, so that nested back-references can be successfully resolved during content deserialization. However, a read-only collection can only be allocated after its contents have been read, since by definition it is read-only.

Arrays, however, are peculiar in that they are only "semi" read-only: they cannot be resized after being allocated, however individual entries can be changed. (see Array.IsReadOnly inconsistent depending on interface implementation for a discussion about this.) It's possible to take advantage of this fact to create a custom JsonConverter for arrays that, during reading, loads the JSON into an intermediate JToken, allocates an array of the correct size by querying the token's contents, adds the array to the serializer.ReferenceResolver, deserializes the contents into a list, then finally populates the array entries from the list:

public class ArrayReferencePreservngConverter : JsonConverter
{
    const string refProperty = "$ref";
    const string idProperty = "$id";
    const string valuesProperty = "$values";

    public override bool CanConvert(Type objectType)
    {
        // byte [] is serialized as a Base64 string so incorporate fix by https://stackoverflow.com/users/13109224/dalecooper from https://stackoverflow.com/a/66177664
        if (objectType == typeof(byte[]))
            return false; // He would kill a Byte[] and you'll wonder, why the JSON Deserializer will return NULL on Byte[] :-)
        // Not implemented for multidimensional arrays.
        return objectType.IsArray && objectType.GetArrayRank() == 1;
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        Debug.Assert(CanConvert(objectType));
        if (reader.TokenType == JsonToken.Null)
            return null;
        else if (reader.TokenType == JsonToken.StartArray)
        {
            // No $ref.  Deserialize as a List<T> to avoid infinite recursion and return as an array.
            var elementType = objectType.GetElementType();
            var listType = typeof(List<>).MakeGenericType(elementType);
            var list = (IList)serializer.Deserialize(reader, listType);
            if (list == null)
                return null;
            var array = Array.CreateInstance(elementType, list.Count);
            list.CopyTo(array, 0);
            return array;
        }
        else
        {
            var obj = JObject.Load(reader);
            var refId = (string)obj[refProperty];
            if (refId != null)
            {
                var reference = serializer.ReferenceResolver.ResolveReference(serializer, refId);
                if (reference != null)
                    return reference;
            }
            var values = obj[valuesProperty];
            if (values == null || values.Type == JTokenType.Null)
                return null;
            if (!(values is JArray))
            {
                throw new JsonSerializationException(string.Format("{0} was not an array", values));
            }
            var count = ((JArray)values).Count;

            var elementType = objectType.GetElementType();
            var array = Array.CreateInstance(elementType, count);

            var objId = (string)obj[idProperty];
            if (objId != null)
            {
                // Add the empty array into the reference table BEFORE poppulating it,
                // to handle recursive references.
                serializer.ReferenceResolver.AddReference(serializer, objId, array);
            }

            var listType = typeof(List<>).MakeGenericType(elementType);
            using (var subReader = values.CreateReader())
            {
                var list = (IList)serializer.Deserialize(subReader, listType);
                list.CopyTo(array, 0);
            }

            return array;
        }
    }

    public override bool CanWrite { get { return false; } }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

The memory efficiency of this approach is not great, so for large collections it would be better to switch to a List<T>.

Then use it like:

var settings = new JsonSerializerSettings
{
    Converters = { new ArrayReferencePreservngConverter() },
    PreserveReferencesHandling = PreserveReferencesHandling.All
};
var a2 = JsonConvert.DeserializeObject<Test[]>(jsonString, settings);

Note the converter is completely generic and works for all single-dimensional arrays. Multidimensional arrays are not implemented.

Sample fiddle showing successful deserialization of nested recursive self-references.

dbc
  • 104,963
  • 20
  • 228
  • 340
  • 1
    Thanks for the answer. It ended up just using `PreserveReferencesHandling.Objects` because I realized that I really don't need a reference to an array itself, just the elements within the index. After I changed it, it worked just fine. I'll go ahead and mark yours as answer after a while here – oscilatingcretin Dec 24 '16 at 17:28
  • 1
    The only issue with this is when you put an array inside of an object definition (compile-time) object. Then objectType.IsArray will not be true and the converter will not be executed. – Arash Jan 16 '17 at 22:56
  • There are a few improvements (not mine): https://localcoder.org/cannot-preserve-reference-to-array-or-readonly-list-or-list-created-from-a-non Unfortunately, it's still unreliable and crashes on complex graphs :( – UserControl May 13 '22 at 09:38
1

I think this code is nice but needs a refine

 var elementType = objectType.IsArray ? objectType.GetElementType() : objectType.GetGenericArguments()[0];

objectType.IsGenericType might be true so, we need to use GetGenericArguments()[0]

JNYRanger
  • 6,829
  • 12
  • 53
  • 81
Egbert Nierop
  • 2,066
  • 1
  • 14
  • 16
  • I don't understand when this would be necessary. `CanConvert(Type objectType)` only returns `true` when `objectType.IsArray` so under what scenario would `objectType.IsArray` be `false` inside `ReadJson()`? – dbc May 26 '23 at 18:49
1

if someone utilizes this converter, might be considering to modify this:

 public override bool CanConvert(Type objectType)
 {
        if (objectType == typeof(byte[]))
            return false; // He would kill a Byte[] and you'll wonder, why the JSON Deserializer will return NULL on Byte[] :-)

        return objectType.IsArray;
 }
DaleCooper
  • 21
  • 1
  • Thanks! I incorporated your fix into my answer, with a link to your answer included. – dbc May 26 '23 at 18:54