Getting an OutOfMemoryException while serialising to JSON?

Question

I am serializing , a MultiDictionary<String,Object>

http://powercollections.codeplex.com/ to json .

It has 618 elements with elements being deeply nested ,i.e. a single Object may have several dictionary like objects in it . I am using JSON.Net

String json = JsonConvert.SerializeObject(json, Newtonsoft.Json.Formatting.Indented);

what am i missing ?

MORE INFO: - This was working fine till i was using dynamic , i had to switch to MultiDictionary to allow multiple properties of the same name . It works for most cases , only when the number of items is large , it breaks .

UPDATE: -

I have been able to get a hold of the Memory consumption but cutting down on some elements that were getting added recursively to each element.

Sam · Answer 1 · 2017-03-15T20:13:09.343

20

Assuming you don't have Circular References - if you can't store the whole thing in memory use a StreamWriter(JsonWriter or TextWriter) in Newtonsoft v4.0.30319

using (TextWriter writer = File.CreateText("LocalJSONFile.JSON"))
{
    var serializer = new JsonSerializer();
    serializer.Serialize(writer, myObject);
}

Use JsonWriter if you are trying to pass the string

StringBuilder sb = new StringBuilder();
StringWriter sw = new StringWriter(sb);

using(JsonWriter writer = new JsonTextWriter(sw))
{
  var serializer = new JsonSerializer();
  serializer.Serialize(writer, myObject);
}

edited Mar 15 '17 at 20:13

answered Aug 16 '16 at 20:21

Sam

1,264
14
19

2

Thanks a lot. After so much headache, finally your first solutions regarding TextWriter is worked. I wish, if I could vote for you 10 times. – Dilip Langhanoja Oct 20 '16 at 12:04
How to pass settings? like PreserveReferencesHandling = PreserveReferencesHandling.Objects, ReferenceLoopHandling = ReferenceLoopHandling.Serialize – Gopu_Tunas Jan 30 '20 at 10:45
Also without settings also, above code is failing even though i have 4 GB RAM available in my computer. – Gopu_Tunas Jan 30 '20 at 11:19
You are trying to save a setting of the function within the large object itself? I would recommend a few things - namely don't do that. Save those values in one object somewhere that is specific to the configuration and have the large JSON just be for data. If you cant do that then I would recommend having two scans of the single object, the first extracts the settings, the second processes the large JSON. – Sam Jan 03 '22 at 22:30

score 13 · Accepted Answer · edited May 23 '17 at 12:26

13

It appears that you're running into Ciruclar Reference that is causing OutOfMemoryException or your objects are simply too large for your memory. Use NDepend to check this.

You might find useful getting the total size of your objects.

edited May 23 '17 at 12:26

Community

1
1

answered Dec 25 '11 at 17:58

Tomislav Markovski

12,331
7
50
72

is there a way of checking the circular reference ? – ashutosh raina Dec 25 '11 at 18:10
1

@ashutoshraina http://www.ndepend.com/Features.aspx#DependencyCycle seems like the preferred tool by everyone – Tomislav Markovski Dec 25 '11 at 18:17

Óscar López · Answer 3 · 2011-12-25T18:14:16.123

It's hard to tell without knowing the exact structure of the objects being serialized, but it's possible that, since the object graph is so big, there might be circular references somewhere (an object which points to an object which in turn points to the first object), creating an infinite loop of serialization.

EDIT :

You may use a tool, like NDepend, to find out where the circular references are. Give the trial version a try.

Getting an OutOfMemoryException while serialising to JSON?

3 Answers3

Linked