6

I have a program(x64) that consumes a large amount of memory. I am running it on win server 2008 R2 SP1 with 48 GB RAM(64 bit) , .net frame work 4.5.

I have also set gcAllowVeryLargeObjects = true in app.config.

When I run the program it consumes 18 GB memory after that it gives exception 

EXCEPTION: System.OutOfMemoryException: Insufficient memory to continue the execution of the program.

   at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
   at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
   at System.Text.StringBuilder.Append(String value)
   at System.Xml.XmlTextEncoder.Write(String text)
   at System.Xml.XmlTextWriter.WriteWhitespace(String ws)
   at System.Xml.XmlElement.WriteElementTo(XmlWriter writer, XmlElement e)
   at System.Xml.XmlNode.get_OuterXml()
   at System.Security.Cryptography.Xml.Utils.PreProcessElementInput(XmlElement e
   lem, XmlResolver xmlResolver, String baseUri)
   at System.Security.Cryptography.Xml.Reference.CalculateHashValue(XmlDocument
   document, CanonicalXmlNodeList refList)
   at System.Security.Cryptography.Xml.SignedXml.BuildDigestedReferences()
   at System.Security.Cryptography.Xml.SignedXml.ComputeSignature()

Its giving the "Insufficient memory" however we still have 30 GB memory free . Is it the limit of .net application or the server that giving me this error.

Storm
  • 684
  • 9
  • 20
Abhijeet Yadav
  • 73
  • 1
  • 4
  • 1
    could it be fragmentation issue, does the error occur if you force GC? Or may be this is it: http://stackoverflow.com/a/7537821/212121 – Giedrius Sep 30 '13 at 06:45
  • Usually OOM thrown by framework doesn't have a stack. Check out `StringBuilder.ExpandByABlock` source code - it throws OOM intentionally in some situation. – mikalai Sep 30 '13 at 20:56

2 Answers2

6

You are running into an internal limitation of the StringBuilder class, it cannot generate a string that has more than int.MaxValue characters. That limitation is a pretty strict one in .NET, gcAllowVeryLargeObjects helps but doesn't solve it. The core issue is that the int type isn't large enough to index the string anymore.

You'll have to write smarter code to solve this problem. That needs to start by you using a StreamWriter instead of a StringWriter. In other words, write to a file instead of memory.

You'll still get to use all of that RAM you have on your machine, files are written to the file system cache first. Your program won't be any slower.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
0

There is one thing that comes to my mind, even if there is no clear evidence in the question about this.

Remember that in BCL there is memory limit on List<T> collections, no matter 32 bit or 64 bit. The maximum memory amount single collection instance can allocate is not more then 2GB on both architectures.

You can have a look on use of your types, expecially for List<T>, if there is some leak happened.

Hope this helps.

Tigran
  • 61,654
  • 8
  • 86
  • 123
  • 1
    gcAllowVeryLargeObjects: On 64-bit platforms, enables arrays that are greater than 2 gigabytes (GB) in total size: http://msdn.microsoft.com/en-us/library/hh285054%28v=vs.110%29.aspx – Giedrius Sep 30 '13 at 06:44