5

I have a 64 bit VB.NET application and want to allocate a buffer > 2GB in size.

In the following code both the "new" and the "ReDim" throw an "OverflowException."

How can I allocate buffers > 2GB when these functions only accept signed 32 bit values?

(Is this possible in C#?)

Edit - I am running WinXP 64 with 4GB of RAM.

Dim width As Long = 19005
Dim height As Long = 32768

Dim buffer() As Byte = New Byte((width * height * 4) - 1) {}

Dim size As Long = (width * height * 4) - 1
ReDim buffer(size)
user79755
  • 2,623
  • 5
  • 30
  • 36
  • 1
    How do you know it's a 64-bit application? How does it show up in the process viewer (if you arrange to have it running long enough)? – Martin v. Löwis Jul 31 '09 at 20:03

4 Answers4

2

Apparently it is not possible to allocate more than 2GB even under 64 bit .net application running on a 64 bit OS.

I find this to be very disappointing and completely without regard for what 64 bit applications and OSs are made for. I am dealing with gigantic images and would like to be able to work with the raw bytes all in RAM at once. Now I have to implement paging algorithms to limit the chunks to 2GB.

Hey Microsoft, hows abouts you fix this in the coming .NET release? Yes, I said fix. That's because it's broken. How do you expect 64 bit applications to take off when you do stupid things like this. (Can you tell that I am annoyed.) Thanks for listening.

Link

http://blogs.msdn.com/joshwil/archive/2005/08/10/450202.aspx

user79755
  • 2,623
  • 5
  • 30
  • 36
  • If you read closely, that isn't exactly true. He even provides 2 viable solutions. – NotMe Jul 31 '09 at 20:46
  • 1
    It *is* true that you can't allocate more than 2GB using the .net new / redim functions. Where do you see that it isn't true. The alternatives are more work and should at least in theory be completely unnecessary. – user79755 Jul 31 '09 at 20:59
  • That "Big Array" class looks suspect to me and I wouldn't use it if you paid me. The custom memory allocator like Paint.NET might be useful. – user79755 Jul 31 '09 at 21:02
2

I think the UnmanagedMemoryStream does what you need. MSDN doc for UnmanagedMemoryStream

I think it's a bad idea, to allocate a huge chunk of memory in a garbage collected environment, since most garbage collectors are optimized for small & short lived object. So using raw memory is generally a better and more performant solution for very large objects.

Paul van Brenk
  • 7,450
  • 2
  • 33
  • 38
  • 1
    It seems to still be limited to 2GB as it takes an "integer" (32 bit signed) argument. – user79755 Jul 31 '09 at 20:22
  • Yes. It takes int32 when reading and writing, but the constructor takes 64bit integers. http://msdn.microsoft.com/en-us/library/dd267517(VS.100).aspx – Paul van Brenk Jul 31 '09 at 20:30
  • 2
    It's also not available for Visual Basic as "Visual Basic does not support APIs that consume or return unsafe types." http://msdn.microsoft.com/en-us/library/zw66yex7.aspx – user79755 Jul 31 '09 at 20:40
0

The following works [in theory] (C# syntax):

Array.CreateInstance(typeof(int[]), 0L);

Edit: Make a type with a fixed-size allocated array of 1GB arrays. You can re-index in the Item property via a shift.

Sam Harwell
  • 97,721
  • 20
  • 209
  • 280
0

You may have to use memory mapped files for this, take a look at the MapViewOfFile function.

Otávio Décio
  • 73,752
  • 17
  • 161
  • 228