How to create an array with more then 300 milions of data

Question

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace testes
{
    class Program
    {
        static void Main(string[] args)
        {
            Int64[] a = new Int64[300000000];
            a[0] = 10;

        }
    }
}

After run this code I receive a System.OutOfMemoryException but my computer has 8 gigas of ram free. Could you please help me?

I am a beginner with C#. I already change the compilation target cpu to x64 thinking that only this would be sufficient. thanks Luc

gcAllowVeryLargeObjects - http://msdn.microsoft.com/en-us/library/hh285054%28v=vs.110%29.aspx — Alexei Levenkov, Mar 12 '14 at 01:40
problem is not just memory, but use of contiguous memory: http://stackoverflow.com/questions/4059363/what-is-a-contiguous-memory-block — DeveloperGuo, Mar 12 '14 at 01:45

score 5 · Answer 1 · answered Mar 12 '14 at 01:42

http://msdn.microsoft.com/en-us/library/ms241064(v=vs.110).aspx

By default, when you run a 64-bit managed application on a 64-bit Windows operating system, you can create an object of no more than 2 gigabytes (GB). However, in the .NET Framework 4.5, you can increase this limit. For more information, see the gcAllowVeryLargeObjects element.

Either split up your array or put this in your config:

<configuration>
  <runtime>
    <gcAllowVeryLargeObjects enabled="true" />
  </runtime>
</configuration>

padawan · Answer 2 · 2014-03-12T02:36:09.423

2

Your first mistake is setting the data type as Int64. The more integers you have, less space you want them to take.

Second, you cannot store even 300 million chars. This is too big. You have two choices when you want to operate on such huge data:

You can create a database and operate on it.
You can write your data in a text file and read the integers whenever you need them.

Both ways are a little bit complex, especially you'll need special methods to access an integer randomly in a text file. e.g. you have to write a method to iterate all the lines until you reach the integer you desire etc.

My suggestion for you would be "use database".

Edit: Your physical memory is different than the memory allocated for your compiler. Also, 300 million 64-bit data is way too large to store as a block.

edited Mar 12 '14 at 02:36

answered Mar 12 '14 at 01:39

padawan

1,295
1
17
45

Perhaps OP uses Int64 because that's the data-type they require? I have 300,000,000 int64 (8 byte values) as 2.24 GB. How do you figure that "300 million 64-bit data is way too large than 8 gigabytes"? It isn't. – spender Mar 12 '14 at 01:58
@spender My bad. I missed a decimal and calculated as 22.4GB. I corrected it now. – padawan Mar 12 '14 at 02:35

How to create an array with more then 300 milions of data

2 Answers2