OutOfMemoryException using a memory intensive project

Question

I have read several SO question/answers about this topic, however they made me even more confused. Here is the deal:

I have an application which reads many images from the disk. Even though I have enough RAM to read that much of bytes, I get System.OutOfMemoryException while reading the data, even before start to process them.

Here is my reading process, basicly:

class Image
{
    float[,] pixels;
}

public static void ReadImages()
{
    List<Image> images = new List<Image>();
    for(int i=0;i<length;i++)
    {
         Image image = ReadImageFromDisk(); // Reads image from the disk.
         if ( Do I need this image? ) // Check whether to store on the memory or not
         {
              images.Add(image);
         }
    }
}

The reason I use List<Image> is basicly I don't know how many Image I am going to store in the memory. I am reading the Image from the disk and then I decide whether to store that image on memory or not.

Making a basic memory calculation:

= 28 (image width) x 28 (image height) x 4 (float) x 500,000 (number of images)
= 1568000000 bytes = 1.568 Gigabyte

I am using 64bit Windows 7 and I have 16 Gigabytes of RAM installed. Why I am getting OutOfMemory exception?

It's not really that important how much RAM the machine has. Is this a 32 bit or a 64 bit application? If it is a 32 bit application, the address space is limited to 4 GB (on 64 bit Windows). If this is a 64 bit application, then the CLR still restricts a single object to 2 GB in size. See http://stackoverflow.com/questions/1087982/single-objects-still-limited-to-2-gb-in-size-in-clr-4-0 — Brian Rasmussen, Feb 12 '13 at 20:28
To be honest, I just opened a `new project` and don't know how to make it `32bit/64bit` from visual studio. I am now checking it and I see at `Build` tab at my `Project Properties` `Platform: Active (x86)`. However, there is not the option to choose `x64`. How to change it? — Sait, Feb 12 '13 at 20:32
Go to Configuration manager in the Build menu. From there you can create additional configurations. — Brian Rasmussen, Feb 12 '13 at 20:34
No, not that one. Project + Properties, Build tab, Platform target setting. Change it to AnyCPU. For VS2012 untick the "Prefer 32-bit" option. — Hans Passant, Feb 12 '13 at 20:59

score 0 · Answer 1 · answered Feb 12 '13 at 20:37

Without knowing precisely what the ReadImageFromDisk function does, my first guess is that you might not be properly disposing of your unmanaged resources, namely those around accessing the filesystem.

If you've got 500k undisposed file system handles floating around (not to mention memory streams used to read the image into an Image object), chances are high for an OOM Exception.

score 0 · Answer 2 · answered Feb 12 '13 at 20:39

Are you targeting your .NET project to build for x86-64bit CPU?

Have you done any debugging to find out exactly where the OOM exception occurs.

Process proc = System.Diagnostics.Process.GetCurrentProcess();
List<Image> images = new List<Image>();
for(int i=0;i<length;i++)
{
    /* DEBUG */
    if((i % 1000) == 0) Console.WriteLine("Image#: " + i + " " + proc.WorkingSet64);

OutOfMemoryException using a memory intensive project

2 Answers2