How much RAM C# application can use?

Question

I have application with very big DataTable objects, very big arrays etc.

Currently my memory usage is below 2GB.

What will happen when my application produce 4 arrays with 1GB size on 32 bit system?

Will it crash?

I know about 2GB object size limit in CLR, but what about many big objects?

I tried to just test it, I declared few big arrays, but when they are empty they seems to not use RAM. I didn't tried to fill them, i decided to just ask here.

Answer 1

64 bit application, including managed ones, can access large memory.

32 bit application have a virtual address space of maximum 4GB. In practaice they get 2GB by default. /LARGEADRESSAWARE applicaitons can get 3GB with /3gb in boot.ini or 4GB when running on WoW64.

Managed application, even on x64 cannot allocate any single object larger than 2GB. Including arrays.

But no matter the amount of VA available at your disposal: manipulating a +2GB DataTable object is just not going to work. Use a storage engine capable of handling and manipulating large amounts of data fast, and capable of intelligent paging. Redis, Cassandra or even traditional RDBMSs are more suited for the job.

Even if you decide to manipulate the data directly in memory, you need a smarter format than DataTable.

Answer 2

All 32bit applications are limited to 2GB of memory*. So no matter how you divvy the RAM up you'll run out at 2GB.

If you need more you'll need to run it on a 64bit system and compile it for 64bit.

• Using LARGEADDRESSAWARE 32bit applications can use up to 3GB of memory, thanks Marc Gravell for reminding me of this in the comments.

Answer 3

If you use more RAM than you can address (or have), you will get an System.OutOfMemoryException exception. try this code to see:

        List<byte[]> foo = new List<byte[]>();
        for (int i = 0; i < 10000; ++i )
            foo.Add(new byte[1024*1024*1024]);

Answer 4

Yes, it will crash. This is the sample code I tried:

class Program
{
    static byte[] arr = new byte[1024 * 1024 * 1024];
    static byte[] arr2 = new byte[1024 * 1024 * 1024];
    static byte[] arr3 = new byte[1024 * 1024 * 1024];
    static byte[] arr4 = new byte[1024 * 1024 * 1024];
    static void Main(string[] args)
    {            
        Console.ReadKey();
    }
}

This is a rather primitive test program. Unless the full amount of data is strongly reachable at once it might already be partially reclaimed by GC and therefore not crash.

You can also make them local. However, if you make a Release Build (with code optimisation) the following bit works like a charm, since unused arrays (or generally variables) may be optimised away:

static void Main(string[] args)
{
    byte[] arr = new byte[1024 * 1024 * 1024];
    byte[] arr2 = new byte[1024 * 1024 * 1024];
    byte[] arr3 = new byte[1024 * 1024 * 1024];
    byte[] arr4 = new byte[1024 * 1024 * 1024];
    Console.ReadKey();
}

In a Debug Build (no code optimisation) the above bit crashes with an OutOfMemoryException too.

Fields in contrast cannot be just left away of course. Therefore the first sample crashes no matter if the code is optimised by the compiler or not.

Answer 5

For the physical memory limits of different operating systems see this article Memory Limits for Windows Releases.

It is possible though, to increase the limit for .NET applications running as 32-bit, from 2GB to up to 3GB. For that you have to run editbin /LARGEADDRESSAWARE on your executable. You can find more details about it here:

• How do I create a (32-bit) .NET application to use 3 GB RAM?
• Increase the memory limit