Simple C# Array throwing OutOfMemory exception

Question

I have a fairly large, fairly simple, array that's tracking 750,000 entries. In 64-bit .NET it works fine; however on the 32-bit client it's failing with an OutOfMemory exception. I know I'm not running out of memory. The client needing this program, cannot use the 64-bit version.

The following pseudo code compiles (edit: the original code did not compile due to a typo, this has been fixed) and, throws the same exception, when targeting the x86 architecture, on two machines:

A Windows XP, 32-bit machine, running 4 GB of RAM and .NET Framework 4
A Windows 7, 64-bit machine, running 8 GB of RAM and .NET Framework 4

Edit: Switching the compiler flag to explicitly to target the x64 architecture allows it to run on the Windows 7, 64-bit machine.

using System;

public class Storage
{
    Storage futureStorage;
    int count;
    int years;
    bool isTest;
    bool isSet;
}

public static class Program
{
    public static void Main()
    {
        Storage[] storageArray = new Storage[750000];
        for (int i = 0; i < 750000; i++)
        {
            storageArray[i] = new Storage();
        }
    }
}

Edit:

The network this code is running on is a closed network, no external network connection, so posting more exact code and relevant exception details is a bit more of a process that I am working on. I do have intellitrace running, with full debug symbols and a few compiler switches, more details to follow as soon as I port it over to an internet connected network.

On both machines, the code bombs out around an index of 693,000 (693,002 on XP, 693,646 on Windows 7 to be exact).

Anyone have any ideas as to what the issue might be? When I do a dump and profile after the code, I'm barely using 25 MB of RAM.

I'd switch to unsafe/unmanaged if it buys me anything, but the only attempt I've made at it makes me unable to use my Storage class.

Update:

Seems several folks are unable to replicate the issue (including myself on a different machine/network).

However, I came up with a solution (using LINQ) that eases the problem, it only throws the exception 1/5 times or so now, anyone know if there's any issues or caveats I should be aware of using this solution, and why internally it might work compared to the other one?

using System;
using System.Linq;    

public class Storage
{
    Storage futureStorage;
    int count;
    int years;
    bool isTest;
    bool isSet;
}

public static class Program
{
    public IEnumerable<Storage> storageEnumerable;
    public static void Main()
    {
        storageEnumerable = (from x in  Enumerable.Range(0, 750000) select new Storage() {count = x});
        foreach (Storage s in storageEnumerable)
            Console.WriteLine(x);
    }
}

The above LINQ code uses far less memory, but the exception still occurs; and only on machines in this network, not on other machines outside of the network (or within an IIS application pool for that matter).

Considering other users have since reported not being able to reproduce the bug, it seems to possibly be environment or hardware specific. Could there be a network monitoring tool, or anti-virus program triggering on these two code implementations? If so, is there a better/recommended way to initialize the array so that I avoid that?

If you used `short count, years;` you could cut your storage nearly in half. — Peter Gluck, Aug 16 '12 at 07:18

score 1 · Answer 1 · edited Jun 20 '20 at 09:12

For the record, from Jeffrey Richter (chapter of thread work):

a 32-bit process has at most 2 GB of usable address space within it. After a bunch of Win32 DLLs load, the CLR DLLs load, the native heap and the managed heap is allocated, there is approximately 1.5 GB of address space left over.

Attempting to create more threads will result in an OutOfMemoryException being thrown.

Of course, a 64-bit process offers 8 terabytes of address space, so you could theoretically create hundreds of thousands of threads

So after you initialize your array, you have less and less free space available. And at certain moment you have no empty memory, and you got your OutOfMemoryException.

I think the LINQ usage more wisely uses resources, and in some times it works without errors.

Unfortunately, you can't do many things with that, sorry about that. The only thing you can do

use not so many objects in your array
use GC.Collect more often to collect objects that are not in use any more

score 1 · Answer 2 · answered Aug 21 '12 at 09:18

1

Try make your Storage struct instead class Array of vlaue types consume less memory that Array of reference types

answered Aug 21 '12 at 09:18

Alex A

141
6

Not possible due to the reference to a type of itself. – leppie May 20 '14 at 14:54

score 1 · Accepted Answer · edited May 23 '17 at 11:51

1

Maybe this will help?

What is the Maximum Size that an Array can hold?

The answer says maybe using List may use less memory.

edited May 23 '17 at 11:51

Community

1
1

answered Aug 22 '12 at 11:00

user1616625

1,072
1
9
15

That article didn't fix the problem, but it did reduce the frequency from 1/5 to about 1/1700 or so; and I've found a way to trap it and retry it, not the most efficient use of resources, but it works, and its generally acceptable. – Brian Deragon Nov 06 '12 at 22:05

score 1 · Answer 4 · answered Dec 14 '11 at 09:07

1

My guess without more debug info is that your program uses virtual memory. Use the taskmanager to explore all memory used, you have to add these fields to your process explorer.If this is the cae you do actually run out of memory.

answered Dec 14 '11 at 09:07

Ruzzie

124
4

Simple C# Array throwing OutOfMemory exception

Update:

4 Answers4