1

I have to serialize an Newtonsoft JObject with DataContractSerializer, it crashes with stack overflow. How to make it work? My code is.

var serializer = new DataContractSerializer(typeof(JObject));
MemoryStream stream1 = new MemoryStream();
var writer = XmlDictionaryWriter.CreateBinaryWriter(stream1);
var obj = new JObject();
serializer.WriteObject(writer, obj);
writer.Flush();

The following example is converting JObject to common type using ISerializationSurrogateProvider functionality. It will crash with stack overflow.


using System;
using System.IO;
using Newtonsoft.Json.Linq;
using System.Runtime.Serialization;
using System.Xml;

class Program
{
    [DataContract(Name = "JTokenReference", Namespace = "urn:actors")]
    [Serializable]
    public sealed class JTokenReference
    {
        public JTokenReference()
        {
        }

        [DataMember(Name = "JType", Order = 0, IsRequired = true)]
        public JTokenType JType { get; set; }

        [DataMember(Name = "Value", Order = 1, IsRequired = true)]
        public string Value { get; set; }

        public static JTokenReference From(JToken jt)
        {
            if (jt == null)
            {
                return null;
            }
            return new JTokenReference()
            {
                Value = jt.ToString(),
                JType = jt.Type
            };
        }
        public object To()
        {
            switch (JType)
            {
                case JTokenType.Object:
                    {
                        return JObject.Parse(Value);
                    }
                case JTokenType.Array:
                    {
                        return JArray.Parse(Value);
                    }
                default:
                    {
                        return JToken.Parse(Value);
                    }
            }
        }
    }

    internal class ActorDataContractSurrogate : ISerializationSurrogateProvider
    {
        public static readonly ISerializationSurrogateProvider Instance = new ActorDataContractSurrogate();

        public Type GetSurrogateType(Type type)
        {
            if (typeof(JToken).IsAssignableFrom(type))
            {
                return typeof(JTokenReference);
            }

            return type;
        }

        public object GetObjectToSerialize(object obj, Type targetType)
        {
            if (obj == null)
            {
                return null;
            }
            else if (obj is JToken jt)
            {
                return JTokenReference.From(jt);
            }

            return obj;
        }

        public object GetDeserializedObject(object obj, Type targetType)
        {
            if (obj == null)
            {
                return null;
            }
            else if (obj is JTokenReference reference &&
                    typeof(JToken).IsAssignableFrom(targetType))
            {
                return reference.To();
            }
            return obj;
        }
    }

    [DataContract(Name = "Test", Namespace = "urn:actors")]
    [Serializable]
    public class Test
    {
        [DataMember(Name = "obj", Order = 0, IsRequired = false)]
        public JObject obj;
    }

    static void Main(string[] args)
    {
        var serializer = new DataContractSerializer(typeof(Test),
        new DataContractSerializerSettings()
        {
            MaxItemsInObjectGraph = int.MaxValue,
            KnownTypes = new Type[] { typeof(JTokenReference), typeof(JObject), typeof(JToken) },
        });

        serializer.SetSerializationSurrogateProvider(ActorDataContractSurrogate.Instance);

        MemoryStream stream1 = new MemoryStream();
        var writer = XmlDictionaryWriter.CreateBinaryWriter(stream1);
        var obj = new JObject();
        var test = new Test()
        {
            obj = obj,
        };
        serializer.WriteObject(writer, test);
        writer.Flush();
        Console.WriteLine(System.Text.Encoding.UTF8.GetString(stream1.GetBuffer(), 0, checked((int)stream1.Length)));
    }
}

I am trying to define a new type JTokenReference to replace JObject/JToken when serializing, but it crashed before replace happens. It seems it failed to resolve the type.

dbc
  • 104,963
  • 20
  • 228
  • 340
tanxin
  • 145
  • 10
  • 1
    I would suggest you first convert the `JObject` into standard .NET types using [this answer](https://stackoverflow.com/a/19140420/10263), then you can serialize that result using the `DataContractSerializer`. – Brian Rogers May 15 '21 at 02:18
  • 1
    Why are you trying to serialize a `JObject` with `DataContractSerializer`? A `JObject` is the JSON document object model of Json.NET, there is no reason to imagine it could be serialized to XML by `DataContractSerializer`. If you just need to convert JSON to XML, see [How to convert JSON to XML or XML to JSON?](https://stackoverflow.com/q/814001/3744182). If you have already parsed to a `JObject` you can use `JsonExtension.ToXElement()` from [How to generate XML from JSON with parent node of array items](https://stackoverflow.com/a/38731234/3744182) to convert it to XML. – dbc May 15 '21 at 14:42
  • I used ISerializationSurrogateProvider to convert it to a .NET types, but it crashed too with the same reason. – tanxin May 17 '21 at 00:59
  • The 3rd-party framework converts every parameter to XML, and unfortunately my parameter is a json.net object. I have no choice, so how to work around this? – tanxin May 17 '21 at 02:04
  • *ISerializationSurrogateProvider to convert it to a .NET types, but it crashed too with the same reason.* - then can you please share a [mcve] with JSON and code that reproduce the problem? Also, was it a **crash**, or an **exception**? If the latter, please also share the full `ToString()` output of the exception including the exception type, message, traceback and inner exception(s) if any. – dbc May 17 '21 at 13:20
  • I edited my question and put my test code to produce the crash. – tanxin May 18 '21 at 02:31

1 Answers1

1

TL;DR

Your approach is reasonable, and ought to work, but fails due to what seems to be a bug in the ISerializationSurrogateProvider functionality with recursive collection types. You're going to need to change your design to use surrogate properties whenever you need to serialize a JToken, e.g. as follows:

[IgnoreDataMember]
public JObject obj { get; set; }

[DataMember(Name = "obj", Order = 0, IsRequired = false)]
string objSurrogate { get { return obj?.ToString(Newtonsoft.Json.Formatting.None); } set { obj = (value == null ? null : JObject.Parse(value)); } }

Explanation

The crash you are experiencing is a stack overflow, and can be reproduced more simply as follows. When the data contract serializer writes a generic such as List<string>, it constructs a data contract name by combining the generic class and parameter names like so:

  • List<string>: ArrayOfstring
  • List<List<string>: ArrayOfArrayOfstring
  • List<List<List<string>>>: ArrayOfArrayOfArrayOfstring

And so on. As the generic nesting gets deeper the name gets longer. Well then, what happens if we define a self-recursive collection type like the following?

public class RecursiveList<T> : List<RecursiveList<T>>
{
}

Well, if we try to serialize one of these list with the data contract serializer, it crashes with a stack overflow exception trying to figure out the contract name. Demo fiddle #1 here -- you will need to uncomment the line //Test(new RecursiveList<string>()); to see the crash:

Stack overflow.
   at System.ModuleHandle.ResolveType(System.Runtime.CompilerServices.QCallModule, Int32, IntPtr*, Int32, IntPtr*, Int32, System.Runtime.CompilerServices.ObjectHandleOnStack)
   at System.ModuleHandle.ResolveTypeHandleInternal(System.Reflection.RuntimeModule, Int32, System.RuntimeTypeHandle[], System.RuntimeTypeHandle[])
   at System.Reflection.RuntimeModule.ResolveType(Int32, System.Type[], System.Type[])
   at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(System.Reflection.MetadataToken, System.Reflection.MetadataImport ByRef, System.Reflection.RuntimeModule, System.Reflection.MetadataToken, System.RuntimeType, Boolean, ListBuilder`1<System.Object> ByRef, System.RuntimeType ByRef, System.IRuntimeMethodInfo ByRef, Boolean ByRef)
   at System.Reflection.CustomAttribute.IsCustomAttributeDefined(System.Reflection.RuntimeModule, Int32, System.RuntimeType, Int32, Boolean)
   at System.Reflection.CustomAttribute.IsDefined(System.RuntimeType, System.RuntimeType, Boolean)
   at System.Runtime.Serialization.CollectionDataContract.IsCollectionOrTryCreate(System.Type, Boolean, System.Runtime.Serialization.DataContract ByRef, System.Type ByRef, Boolean)
   at System.Runtime.Serialization.CollectionDataContract.IsCollectionHelper(System.Type, System.Type ByRef, Boolean)
   at System.Runtime.Serialization.DataContract.GetNonDCTypeStableName(System.Type)
   at System.Runtime.Serialization.DataContract.GetStableName(System.Type, Boolean ByRef)
   at System.Runtime.Serialization.DataContract.GetCollectionStableName(System.Type, System.Type, System.Runtime.Serialization.CollectionDataContractAttribute ByRef)
   at System.Runtime.Serialization.DataContract.GetNonDCTypeStableName(System.Type)
   at System.Runtime.Serialization.DataContract.GetStableName(System.Type, Boolean ByRef)
   at System.Runtime.Serialization.DataContract.GetCollectionStableName(System.Type, System.Type, System.Runtime.Serialization.CollectionDataContractAttribute ByRef)
   at System.Runtime.Serialization.DataContract.GetNonDCTypeStableName(System.Type)
   at System.Runtime.Serialization.DataContract.GetStableName(System.Type, Boolean ByRef)

Oops. Well, what if we create a serialization surrogate such as the following dummy surrogate for RecursiveList<string>

public class RecursiveListStringSurrogate
{
    // A dummy surrogate that serializes nothing, for testing purposes.
}

public class RecursiveListStringSurrogateSelector : ISerializationSurrogateProvider
{
    public object GetDeserializedObject(object obj, Type targetType)
    {
        if (obj is RecursiveListStringSurrogate)
            return new RecursiveList<string>();
        return obj;
    }

    public object GetObjectToSerialize(object obj, Type targetType)
    {
        if (obj is RecursiveList<string>)
            return new RecursiveListStringSurrogate();
        return obj;
    }

    public Type GetSurrogateType(Type type) 
    {
        if (type == typeof(RecursiveList<string>))
            return typeof(RecursiveListStringSurrogate);
        return type;
    }
}

Using that surrogate, an empty new RecursiveList<string>() can indeed be serialized successfully, as

<RecursiveListStringSurrogate xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/" />

Demo fiddle #2 here.

OK, now let's try using the surrogate when a RecursiveList<string> is embedded in a model such as:

public class Model
{
    public RecursiveList<string> List { get; set; }
}

Well when I try to serialize an instance of this model with an empty list, the crash comes back. Demo fiddle #3 here - you will need to uncomment the line //Test(new Model { List = new RecursiveList<string>() }); to see the crash.

Oops again. It's not entirely clear why this fails. I can only speculate that, somewhere, Microsoft is keeping a dictionary mapping original data contract names to surrogate data contract names -- which causes a stack overflow simply generating a dictionary key.

Now what does this have to do with JObject and your Test class? Well it turns out that JObject is another example of a recursive collection type. It implements IDictionary<string, JToken?> and JToken in turn implements IEnumerable<JToken> thereby triggering the same stack overflow we saw with the simple model containing a RecursiveList<string>.

You might even want to report an issue to Microsoft about this (though I don't know whether they are fixing bugs with the data contract serializer any more.)

Workaround

To avoid this issue, you will need to modify your model(s) to use surrogate properties for JToken members as shown at the beginning of this answer:

[DataContract(Name = "Test", Namespace = "urn:actors")]
public class Test
{
    [IgnoreDataMember]
    public JObject obj { get; set; }
    
    [DataMember(Name = "obj", Order = 0, IsRequired = false)]
    string objSurrogate { get { return obj?.ToString(Newtonsoft.Json.Formatting.None); } set { obj = (value == null ? null : JObject.Parse(value)); } }
}

Which can be serialized successfully as follows:

var obj = new JObject();
var test = new Test()
{
    obj = obj,
};

var serializer = new DataContractSerializer(test.GetType());

MemoryStream stream1 = new MemoryStream();
var writer = XmlDictionaryWriter.CreateBinaryWriter(stream1);
serializer.WriteObject(writer, test);
writer.Flush();
Console.WriteLine(System.Text.Encoding.UTF8.GetString(stream1.GetBuffer(), 0, checked((int)stream1.Length)));

Notes:

  • If you need to serialize a JToken as the root object you can either wrap it in some container object, or use the ActorDataContractSurrogate from your question. As we have seen, the serialization functionality does seem to work for recursive collection types when they are the root object.

  • Since you are serializing to binary, for efficiency I suggest formatting the JObject with Formatting.None.

  • The surrogate property can be private as long as it is marked with [DataMember].

Demo fiddle #4 here.

dbc
  • 104,963
  • 20
  • 228
  • 340