Why do I get intermittent System.OutOfMemoryException?

Question

Why does the same code sometimes run, and sometimes give me an exception?

I am just running a test, looking at the limits of strings in memory. The entirety of my Main.vb reads:

Module Main
Sub Main()
    teststring()
End Sub
Sub teststring()
    Dim strC As String
    strC = New String("C"c, 588060752)
End Sub
End Module

It will run just fine sometimes (usually but not always when I haven't run it in a minute or so), and other times will fail with an exception.

Because it depends on how much available memory there is at the time http://stackoverflow.com/questions/4486261/does-c-sharp-string-has-length-limit — Slai, Jan 23 '17 at 21:59
@Slai While your comment makes sense (that OutOfMemory means Out of Memory), your linked question doesn't address available memory. It is a duplicate of another question that doesn't address available memory. As a matter of fact, both questions imply that the upper limit of string size is a fixed number, or at least fixed on any given machine. Do you have any suggestions for a different link/duplicate target for me? — CWilson, Jan 23 '17 at 22:13

score 3 · Accepted Answer · answered Jan 24 '17 at 03:35

3

"Out of memory" can occur when

You literally don't have enough memory, e.g. you requested a string that would consume a trillion bytes. I don't think you can buy a server with terabytes of RAM, not yet. Actually, I believe the CLR limits large objects to 2 GB anyway.
You have enough memory, but some of it is in use by other processes, or by other objects in your process, so there isn't enough free memory.
You have enough free memory, but the heap is fragmented and there isn't enough room in any one spot to allocate contiguous memory.
You exceeded a hard system limit (e.g. allocating a StringBuilder beyond its maximum size).

These are the big reasons. More specifics can be found in the documentation for OutOfMemoryException.

answered Jan 24 '17 at 03:35

John Wu

Thank you. The doc pages and other SO questions I had read didn't mention #3, and I had assumed that automatic GC at the end of my process would automatically defrag my memory. I can't imagine an explicit `GC.Collect` at the end of my code would do any good here, right? Any suggestions for reading material on how to defrag the heap in .net as the first subroutine in an application? Maybe that should be another question. – CWilson Jan 24 '17 at 04:25
Hmm... I added `System.Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce` `GC.Collect()` before and after the `teststring()` in the `Sub Main()`, and I still get OOMEx, once every 3 or 4 times, sometimes a few times in a row. Doesn't sound like a defragmented heap after all. – CWilson Jan 24 '17 at 04:59
It could be due to ASLR choosing different addresses for things and sometimes fragmenting the memory in a way that prevents you from reserving the entire block. Garbage Collection in a previous run of the application should have absolutely no effect on subsequent runs. – Bradley Uffner Jan 24 '17 at 05:04
Have you tried setting `gcAllowVeryLargeObjects` in the config file? https://msdn.microsoft.com/en-us/library/hh285054(v=vs.110).aspx – Bradley Uffner Jan 24 '17 at 05:07
My post includes a link to an article on debugging fragmented heaps. – John Wu Jan 24 '17 at 06:11

1 Answers1