0

I'm experimenting with memory limits on my x64 machine (Windows7) under .NET 4.5. Based on this documentation (as referred to in this answer) it seems like I should be able to create an array with up to UInt32.MaxValue elements. So I enabled gcAllowVeryLargeObjects and tried to create the following array:

int[] arr = new int[UInt32.MaxValue]; //2^32

However, this generates an OverflowException: ("Arithmetic operation resulted in an overflow.")

So I tried this instead,

int[] arr = new int[UInt32.MaxValue/2];   //2^31  (i.e., 2 * 1024 * 1024 * 1024)

and it throws a 'System.OutOfMemoryException'.

What am I missing here, and what in fact is the largest integer array that I can create?

UPDATE: So, as it turns out, the largest array I can create is:

int[] arr = new int[(long)(1.999 * 1024 * 1024 * 1024)]; //just shy of 2^31

So why can't I reach the limit that the documentation leads me to expect?

Community
  • 1
  • 1
kmote
  • 16,095
  • 11
  • 68
  • 91
  • Did you thought that "the fact" can not be reproduced in your environment? – Leandro Bardelli Nov 05 '14 at 23:56
  • 1
    At run time it would depend on how much contiguous addressable memory you have available, and the size of the thing in the array. – Preston Guillot Nov 05 '14 at 23:56
  • I'm pretty sure windows, by default, limits the program to 2GB of memory. I think there are ways around that. – MikeH Nov 06 '14 at 00:05
  • 1
    @MikeH: You're thinking of 32-bit Windows, the Q clearly says x64. And the 64-bit .NET compilers certainly don't do anything so stupid as clearing the `/LARGEADDRESSAWARE` flag in the PE header (which would limit even a 64-bit program to 2GB) – Ben Voigt Nov 06 '14 at 00:05
  • @BenVoigt ...I must learn to read... – MikeH Nov 06 '14 at 00:06

1 Answers1

4

From your link:

The maximum index in any single dimension is 2,147,483,591 (0x7FFFFFC7) for byte arrays and arrays of single-byte structures, and 2,146,435,071 (0X7FEFFFFF) for other types.

You're exceeding that limit, still.

This should give a darn big array, though:

int[,] arr = new int[UInt32.MaxValue/3,3];
//                 comes out even ^^^

The total size of that array, which contains the maximum number of elements, should be (16 GB - 4 bytes + metadata) Use an even larger value type to really put the hurt on your PC.

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • you would think that the compiler would object to int[uint.maxvalue] since it is clearly wrong – pm100 Nov 06 '14 at 00:09
  • @pm100: There is, of course, cost associated with adding a compile error for that. As Eric Lippert likes to remind us, a specification has to be written and approved, the code has to be written, and then it has to be tested. Also, the compiler hasn't any way to know whether `gcAllowVeryLargeObjects` will be in effect when the program runs, so it can't actually know what limit to enforce. A much better option would be a clear message in the exception... it could say what is exceeded, the overall array size, the total number of elements, or the size of a single dimension. – Ben Voigt Nov 06 '14 at 00:11
  • @pm100 is it actually wrong according to the C# specification or the CLS specification? It seems more like this is a current implementation limitation. Array.LongLength is there for 64 bit array indexes after all. – usr Nov 06 '14 at 00:19
  • Excellent, Ben. I just didn't read the docs close enough, I guess. Brilliant example with the 2D array to clarify the distinction between the number of elements and the size of the index. THank you! – kmote Nov 06 '14 at 00:39