Array declared with maximum size throws OutOfMemoryException in C#

Question

I understand a similar question has been asked many times and answered as well. Much of the articles on internet point to only one answer i,e. max of int value or 2147483647.

Inline are couple of links from stackoverflow:

What is the Maximum Size that an Array can hold?

Maximum size of string array in C#

However, when I tried to check the same an OutOfMemoryException is thrown. In fact I tried to reduce the size by a factor of 2 to 1073741823(throws exception), factor of 4 to 536870911(throws exception), factor of 8 to 268435455 (program executes well without any exception).

Test Code:

using System;

namespace ConsoleApp
{
    class Program
    {
        static void Main()
        {
            int arraySize = Int32.MaxValue / 1; //Factor when set to 8, program executes without any exception
            int[] testArray = new int[arraySize];

            testArray[0] = 6;

            Console.WriteLine("Press Enter to finish.");
            Console.ReadLine();
        }
    }
}

Looks like all the other answers point to a theoretical value which is `2147483647. However in practice this is not always the case. So the question is, given the application code is very simple with just one array and couple of calls to console, is it possible to figure out at what size of array the application will throw an OutOfMemoryException.

Which .NET version? What bitness? Do you have `gcAllowVeryLargeObjects`? The point is that your application needs to be able to allocate a contiguous amount of arraySize*elementSize of RAM. If it can't, it'll fail. AFAIK it's not trivial using managed code to predict whether that will succeed. What do you need that large an array for anyway? — CodeCaster, Aug 12 '20 at 10:11
Int32.MaxValue = 2147483647 which is larger than the max size of an array. — jdweng, Aug 12 '20 at 10:14
That first question you linked gives you the answer you need, did you read it all? — DavidG, Aug 12 '20 at 10:15
Ultimately, and I *cannot emphasize this enough*: having huge vectors is *usually* a bad idea and is going to hurt you, so asking "what is the largest array that I can theoretically have" is *usually* a sign that someone is (metaphorically) pointing a shotgun at their foot, and asking what the most powerful shell they can theoretically use is. There *are* some scenarios where it is relevant, but they are niche, and often borrowing unmanaged memory (perhaps using `Memory`/`Span`) is a better idea. — Marc Gravell, Aug 12 '20 at 10:16
@CodeCaster The .NET Version i have checked with is 4.7.2 and .net core 3.1 (64 bit OS). I do not have gcAllowVeryLargeObjects. The reason i asked it was I was trying to understand if really it was possible to declare such a big array, and in reality if it isn't possible why no one has put a disclaimer around it. — Durga Prasad, Aug 12 '20 at 10:19
@DurgaPrasad if you don't have gcAllowVeryLargeObjects, then very clearly: no you won't - but this is very well documented — Marc Gravell, Aug 12 '20 at 10:22
If you try an array size of something like 536,870,850, that will probably work (it puts you just below the 2GB limit for single objects). However you can't guarantee that it will work, as the OS might not be able to allocate you enough contiguous memory. There's no guarantee that *any* allocation will succeed, but the likelihood of failure goes up the larger the allocation — canton7, Aug 12 '20 at 10:22
@DavidG The first question answer states that you can declare an array with size equivalent to int.maxvalue. When i tried it, it does not work. In fact I am off by a factor of 8. Am I missing something in the answer. — Durga Prasad, Aug 12 '20 at 10:23
@DurgaPrasad See e.g. [this answer](https://stackoverflow.com/a/23251007/1086121) to the first question you linked, and [this accepted answer](https://stackoverflow.com/a/23251007/1086121) to the second question. The maximum size for any object in .NET is 2GB (unless you specify gcAllowVeryLargeObjects) — canton7, Aug 12 '20 at 10:24
@MarcGravell I will try by setting gcAllowVeryLargeObjects once. — Durga Prasad, Aug 12 '20 at 10:24
_"The first question answer states that you can declare an array with size equivalent to int.maxvalue"_ - that's because that answer is simply looking at the data type of the indexer, which is a gross oversimplification and theoretical at best. — CodeCaster, Aug 12 '20 at 10:25
@canton7 Thank you canton. That was the answer I was looking for. — Durga Prasad, Aug 12 '20 at 10:51

InBetween · Accepted Answer · 2020-08-12T10:22:04.423

2

The NET framework limits the maximum size of any object to 2 GB by default. Arrays can be bigger on 64-bit platforms if you enable the corresponding setting gcAllowVeryLargeObjects.

So theoretically, the limit of an int array should be around 536 870 912 elements, but that doesn't account for the memory footprint of the array itself, therefore the real limit must be less.

Another issue is that arrays need to be allocated in contiguous memory space. If the allocator can't find any such space, you will get an OutOfMemoryException even if the object is under the maximum size limit.

edited Aug 12 '20 at 10:22

answered Aug 12 '20 at 10:13

InBetween

32,319
3
50
90

Thank you @InBetween. The "536 870 912" makes sense for me. I missed out on the math of 2GB must be divided 4. That is a valuable point for me. – Durga Prasad Aug 12 '20 at 10:49

Array declared with maximum size throws OutOfMemoryException in C#

1 Answers1