Language: C++
System: Windows 7 x64
Memory: 8GB RAM
I wanna new a large one-dimensional array in my 64-bit application, which contains 60000*60000=3600000000 unsigned short type elements.
Purpose to read a very big picture which is 60k*60k pixels and turn that into one-demensional array to take further process. I certainly could split that picture and read them separately , but in production environment, i do have 128GB and more RAM to consume.
1. static method
unsigned short array [3600000000];
it shows error: size of array 'a' is negative
2.malloc approach:
unsigned long long bytes = 3600000000 * sizeof(unsigned short);
unsigned short *arr;
arr = (unsigned short *)malloc(bytes);//almost 6.7GB memory
In my pc with 8GB RAM, address of arr is 0x0 when debug at line malloc
in other pc with 16GB RAM, the address of arr is valid, but if i assign value to each item in arr like
#include <iostream>
#include <string.h>
using namespace std;
int main()
{
unsigned short *arr;
arr = (unsigned short *)malloc(3600000000 * sizeof(unsigned short));//memory space can be allocate to arr, about 6.7GB
if (arr == NULL){
cout << "failed"<< endl;
}
memset(arr, 1, sizeof(arr));
cout << arr << endl;
}
Interruption will occured at some weird memory location arr is 0x11103630A52B112.
in my pc x64 with 8GB RAM, it print "failed"
in other pc x64 with 16GB RAM, the address of arr is valid, but if i assign value to each item in arr with for loop, interruption will occured at some wired memory location arr is 0x11103630A52B112
how could i new a very very big one-dimentional array
1.(static method 8GB) it shows error: size of array 'a' is negative
2.(malloc approach 8GB)error log with my 8GB and memset statement
oneDimensionalArray.cpp: In function 'int main()':
oneDimensionalArray.cpp:10:47: warning: unsigned conversion from 'long long int' to 'size_t' {aka 'unsigned int'} changes value from '7200000000' to '2905032704' [-Woverflow]
arr = (unsigned short *)malloc(3600000000 * sizeof(unsigned short));//memory space can be allocate to arr, about 6.7GB
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
oneDimensionalArray.cpp:10:35: warning: argument 1 value '2905032704' exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
arr = (unsigned short *)malloc(3600000000 * sizeof(unsigned short));//memory space can be allocate to arr, about 6.7GB
~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\cstdlib:75,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\ext\string_conversions.h:41,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\bits\basic_string.h:6391,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\string:52,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\bits\locale_classes.h:40,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\bits\ios_base.h:41,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\ios:42,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\ostream:38,
from folderPath\mingw\installed\lib\gcc\mingw32\8.2.0\include\c++\iostream:39,
from oneDimensionalArray.cpp:1:
folderPath\mingw\installed\include\stdlib.h:503:40: note: in a call to allocation function 'void* malloc(size_t)' declared here
_CRTIMP __cdecl __MINGW_NOTHROW void *malloc (size_t) __MINGW_ATTRIB_MALLOC;
^~~~~~
failed
3.(malloc approach 16GB) An exception was raised: write access rights conflict, arr is 0x11103630A52B112 at assign statament arr[i] = 1; in for loop