Serializing an object using Json.Net causes Out of Memory exception

Question

Disclaimer: I did went through most of the solution provided here but most of them were talking about OOM exception while Deserialization.

I am trying to serialize an object( it's a Tree) into Json using Json.Net. Everything works fine for small objects but i get OOM exception when i try it with large objects. As it works with smaller object of same datatype i am assuming there is no circular reference (I did inspect my data structure for it). Is there a way where i can convert my object into stream ( this is a Windows Store app ) and generate the Json using that stream ?

 public static async Task<bool> SerializeIntoJson<T>(string fileName, StorageFolder destinationFolder, Content content)
    {
        ITraceWriter traceWriter = new MemoryTraceWriter();
        try
        {

            string jsonString = JsonConvert.SerializeObject(content, Formatting.Indented, new JsonSerializerSettings
            {
                PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                TypeNameHandling = TypeNameHandling.All,
                Error = ReportJsonErrors,
                TraceWriter = traceWriter,
                StringEscapeHandling = StringEscapeHandling.EscapeNonAscii
            });
            System.Diagnostics.Debug.WriteLine(traceWriter);

            StorageFile file = await destinationFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
            await Windows.Storage.FileIO.WriteTextAsync(file, jsonString);
            return true;
        }
        catch (NullReferenceException nullException)
        {
            System.Diagnostics.Debug.WriteLine(traceWriter);
            logger.LogError("Exception happened while serializing input object, Error: " + nullException.Message);
            return false;
        }
        catch (Exception e)
        {
            System.Diagnostics.Debug.WriteLine(traceWriter);
            logger.LogError("Exception happened while serializing input object, Error: " + e.Message, e.ToString());
            return false;
        }
    }

In order to convert my object into stream, the code i found out was using a BinaryFormatter which is not available in Windows store app dll's.

Try using a `JsonSerializer` combined with a `StreamReader` as outlined [here](http://stackoverflow.com/questions/8157636/can-json-net-serialize-deserialize-to-from-a-stream). — Nate Diamond, Sep 30 '14 at 19:18
I did looked at the code and that brought me to the question : Is there a way where i can convert my object into stream ( this is a Windows Store app )? — AGCodes, Sep 30 '14 at 19:35
That's what the Serializer does. It takes all of the properties and converts them into strings. In this case instead of returning strings it can deposit them directly into a stream (such as a file stream) so that they are not sitting in memory. — Nate Diamond, Sep 30 '14 at 19:42
Ok this worked, i had the TraceListner which was creating strings internally with each serialization and deserialization. — AGCodes, Sep 30 '14 at 20:50
Maybe Nate Diamond could add an answer so that it can be accepted. Worked for me too! — ProfNimrod, Dec 19 '14 at 00:11

score 11 · Answer 1 · edited May 27 '22 at 13:09

It is due to the large amount of records you are trying to serialize, which occupies a large amount of memory. Solutions which I have found for this error is to directly write to the documents using StreamWriter(JsonWriter or TextWriter).

If you have Object use TextWriter:

using (TextWriter textWriter = File.CreateText("LocalJsonFile.json"))
{
    var serializer = new JsonSerializer();
    serializer.Serialize(textWriter, yourObject);
}

If you have a string, use StringWriter:

StringBuilder sb = new StringBuilder();
StringWriter sw = new StringWriter(sb);
    
using (JsonWriter textWriter = new JsonTextWriter(sw))
{
    var serializer = new JsonSerializer();
    serializer.Serialize(textWriter, yourObject);
}

score 6 · Answer 2 · answered Dec 31 '15 at 09:36

Updated Code based on suggestions in the comments on the question, This works!

public static async Task<bool> SerializeIntoJson<T>(string fileName, StorageFolder destinationFolder, Content content)
    {
        try
        {
            StorageFile file = await destinationFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
            using (var stream = await file.OpenStreamForWriteAsync())
            {

                StreamWriter writer = new StreamWriter(stream);
                JsonTextWriter jsonWriter = new JsonTextWriter(writer);
                JsonSerializer ser = new JsonSerializer();

                ser.Formatting = Newtonsoft.Json.Formatting.Indented;
                ser.PreserveReferencesHandling = PreserveReferencesHandling.Objects;
                ser.TypeNameHandling = TypeNameHandling.All;
                ser.Error += ReportJsonErrors;

                ser.Serialize(jsonWriter, content);

                jsonWriter.Flush();

            }
            return true;
        }
        catch (NullReferenceException nullException)
        {

            logger.LogError("Exception happened while serializing input object, Error: " + nullException.Message);
            return false;
        }
        catch (Exception e)
        {

            logger.LogError("Exception happened while serializing input object, Error: " + e.Message, e.ToString());
            return false;
        }
    }

Serializing an object using Json.Net causes Out of Memory exception

2 Answers2

Linked

Related