Difference between actual OOM and 2GB object OOM?

Question

I was wondering if there is some difference between the OOM exceptions thrown for an actual OOM (memory runs out), and the exception thrown when the 2GB object limit is hit?

I have the following code for causing the OOMs (no app.config changes, gcAllowVeryLargeObjects is by default set to false):

struct Data
{
    double a;
    double b;
}

// Causes OOM due to the 2GB object limit.
List<Data> a = new List<Data>(134217725);

// Causes OOM due to actual OOM.
List<Data[]> b = new List<Data[]>();
for (int i = 0; i < 13421772; i++)
{
    b.Add(new Data[134217724]);
}

Now, I've executed the code from Visual Studio, and I get the following exceptions:

2GB object limit

System.OutOfMemoryException
  HResult=0x8007000E
  Message=Exception of type 'System.OutOfMemoryException' was thrown.
  Source=mscorlib
  StackTrace:
   at System.Collections.Generic.List`1..ctor(Int32 capacity) 
   at ConsoleApp1.Program.Main(String[] args)

actual OOM

System.OutOfMemoryException
  HResult=0x8007000E
  Message=Exception of type 'System.OutOfMemoryException' was thrown.
  Source=ConsoleApp1
  StackTrace:
   at ConsoleApp1.Program.Main(String[] args)

From here, it doesn't seem like there is a significant difference between the two exceptions (other than the stack trace/source).

On the other hand, I executed the exact same thing from LINQPad and got the following:

2GB limit

actual OOM

Executing RuntimeInformation.FrameworkDescription from both places results in .NET Framework 4.8.4341.0

My question is about detecting/differentiating between the two cases, although I am also curious as to why the error messages differ between the LINQPad and VS executions.

Do you have a purpose with this question, other than academic curiosity? What versions of .net are you targeting with VisualStudio & linqPad? — JonasH, Mar 05 '21 at 14:03
Happens [here](https://github.com/dotnet/runtime/blob/dc73562d3d057547eaba8ff8313b4c58a157f48d/src/coreclr/vm/gchelpers.cpp#L181) and [here](https://github.com/dotnet/runtime/blob/dc73562d3d057547eaba8ff8313b4c58a157f48d/src/coreclr/vm/gchelpers.cpp#L388). The number of elements may overflow and become negative. Hmm. — Hans Passant, Mar 05 '21 at 14:31
@JonasH the .net versions are both `.NET Framework 4.8.4341.0`. I was wondering if the different cases could somehow be detected, as they sort of represent different states: one cannot be helped (actual OOM) and presents a problem with the application design, while the other (2GB size limit) could be solved by differently handling a single data structure (maybe even during runtime if the difference between exception can be detected). — np_6, Mar 05 '21 at 14:48
Does this answer your question? [How does catching an OutOfMemoryException work?](https://stackoverflow.com/questions/13835778/how-does-catching-an-outofmemoryexception-work) — Sinatr, Mar 05 '21 at 14:58
You can catch OOM and try to allocate again in chunks as you propose (or how I understood it). It will then either fail again with OOM or work. — Sinatr, Mar 05 '21 at 14:59
If you are running 32-bit code you might also have to worry about Large object heap fragmentation. So it might be possible to to allocate two 0.5Gb chunks where 1Gb chunk failed. If you need to handle objects it seem much more reasonable to simply enable `gcAllowVeryLargeObjects`. That said, I wish the exception included the amount of memory it tried to allocate, since this would be useful when investigating exceptions. — JonasH, Mar 05 '21 at 15:02

Matthew Watson · Accepted Answer · 2021-03-05T14:10:41.213

I can explain the difference between LinqPad and Visual Studio:

If you run x86 DEBUG and RELEASE .Net Framework 4.8 builds of the following code:

static void Main()
{
    try
    {
        List<Data> a = new List<Data>(134217725);
    }
    
    catch (Exception e)
    {
        Console.WriteLine(e.Message);
    }

    try
    {
        List<Data[]> b = new List<Data[]>();

        for (int i = 0; i < 13421772; i++)
        {
            b.Add(new Data[134217724]);
        }
    }

    catch (Exception e)
    {
        Console.WriteLine(e.Message);
    }
}

For the RELEASE build you get:

Array dimensions exceeded supported range.
Exception of type 'System.OutOfMemoryException' was thrown.

For the DEBUG build you get:

Exception of type 'System.OutOfMemoryException' was thrown.
Exception of type 'System.OutOfMemoryException' was thrown.

This implies that the LINQPAD version is RELEASE and the Visual Studio version is DEBUG.

So the answer is: Yes, there clearly is some difference, but:

Only the message differs
We should not rely on this never changing
It differs between DEBUG and RELEASE builds.

Aside:

On my PC, the DEBUG build of the test code above immediately throws the two OutOfMemoryException exceptions.

However, the RELEASE build quickly throws the first OutOfMemoryException, but it is several seconds before it throws the second exception. During this time, its memory usage increases (according to Task Manager).

So clearly there is some other difference under the hood, at least for .Net Framework 4.8. I haven't tried this with .Net 5 or .Net Core.

Thanks, it hadn't occurred to me that `DEBUG` and `RELEASE` builds could have different error messages for the same exception. I was able to get the different error messages in VS by switching to `RELEASE`, although worth noting I also had to target specific platforms (didn't work for `RELEASE` + `Any CPU`). — np_6, Mar 05 '21 at 14:36

Difference between actual OOM and 2GB object OOM?

1 Answers1