.NET JSON.decode() fails for large arrays

Question

MVC 4 ASP.NET Web API application :

I have a JSON array of form :

var json = "{
    'mapset':
    [
        {
            'id': '1',
            'key': 'key1',
            'value': 'value1',
            'timestamp': '2014-02-12T08:50:54.594Z'
        },
        {
            'id': '2',
            'key': 'key2',
            'value': 'value2',
            'timestamp': '2014-02-12T08:50:54.594Z'
        },
    ]
}";

dynamic data = System.Web.Helpers.JSON.decode(json);

For array with 10K elements of ~1K bytes each, JSON.Decode() works like a charm.

For 100K elements fails with error :

System.ArgumentException: Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property.Parameter name: input at System.Web.Script.Serialization.JavaScriptSerializer.Deserialize(JavaScriptSerializer serializer, String input, Type type, Int32 depthLimit) at System.Web.Helpers.Json.Decode(String value)

How can I set the limit higher ?

I am aware of the other post regarding this topic :

Can I set an unlimited length for maxJsonLength in web.config?

However the answers provided there do not solve my specific problem.

I am directly using JSON.Decode() within an MVC4 app and so modification of web.config settings will not apply. And JSON.Decode() performs deserialization of a JSON string to a .NET JSON object ( not serialization of a .NET JSON object to a JSON string ).

Yes I know this is a config issue but the other post does not provide a solution. Please see my edits in original question. Please reopen. — BaltoStar, Aug 07 '14 at 04:14

score 2 · Answer 1 · answered Apr 26 '19 at 17:50

This blog article by Michael Witty shows how to solve the problem. Basically, while it is not possible to directly override the MaxJsonLength of the underlying JavaScriptSerializer used by System.Web.Helpers.Json.Decode, one can very easily replicate System.Web.Helpers.Json.Decode with a few lines and there-by control the max length.

The following is a tested and working DynamicJsonObjectFormatter. It has a static method dynamic DynamicJsonObjectFormatter.Decode(string json) that duplicates System.Web.Helpers.Json.Decode, but gives you control of the max length:

public class DynamicJsonObjectFormatter : BufferedMediaTypeFormatter
{
    private static readonly log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

    public static dynamic Decode(string json, int maxLength=0)
    {
        try
        {
            if (string.IsNullOrEmpty(json))
                return null;
            var serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
            if( maxLength>0) {
                serializer.MaxJsonLength = maxLength;
            }
            var deserialized = serializer.DeserializeObject(json);
            if (deserialized != null)
            {
                var dictValues = deserialized as IDictionary<string, object>;
                if (dictValues != null)
                    return new DynamicJsonObject(dictValues);
                var arrayValues = deserialized as object[];
                if (arrayValues != null)
                {
                    return new DynamicJsonArray(arrayValues);
                }
            }
            log.Error("Internal: Attempt to deserialize unrecognized JSON string as DynamicJsonObject");
        }
        catch (Exception ex)
        {
            log.Error("Internal: exception deserializing JSON", ex);
        }
        return null;
    }
    override public object ReadFromStream(Type type, System.IO.Stream readStream, System.Net.Http.HttpContent content, IFormatterLogger formatterLogger)
    {
        System.IO.StreamReader strdr = null;
        try
        {
            strdr = new System.IO.StreamReader(readStream);
            string json = strdr.ReadToEnd();
            int maxLength = 33554432; //Int32.MaxValue;
            return Decode(json, maxLength);
        }
        catch (Exception ex)
        {
            log.Error("Internal: exception deserializing JSON", ex);
        }
        finally
        {
            if (strdr != null)
            {
                strdr.Dispose();
            }
        }
        return null;
    }

    public DynamicJsonObjectFormatter()
    {
        SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/json"));
        SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/json"));
        //SupportedEncodings.Add(new UTF8Encoding(false, true));
    }

    public override bool CanReadType(Type type)
    {
        return (type == typeof(DynamicJsonObject)) || (type == typeof(DynamicJsonArray));
    }

    public override bool CanWriteType(Type type)
    {
        return false;
    }

}

(Note: the strdr.ReadToEnd() may also give you an issue with large files. It's size limit can be controlled with

<system.web>
    <httpRuntime targetFramework="4.6.2" maxRequestLength="200000" />
    <!-- maxRequestLength is in KB, not B -->
</system.web>

in your Web.config file. See this article.)

score 0 · Answer 2 · answered Oct 28 '14 at 09:04

System.Web.Script.Serialization.JavaScriptSerializer does the trick for me.

It is ugly I think, but with the JavaScriptSerializer, I'm able to set the maxJsonLength to a big value:

JavaScriptSerializer ser = new JavaScriptSerializer();
ser.MaxJsonLength = Int32.MaxValue; // <-- should probably not use that huge value
var jsonArtikel = ser.Serialize(Model);

.NET JSON.decode() fails for large arrays

2 Answers2