2

I am writing a program to do some analysis on DNA sequences. Everything works fine except for this thing. I want to declare a 2D array of size m*n where m and n are read from an input file. Now the issue is that if m and n goes too large. As an example if m = 200 and n = 50000 then I get a seg fault at the line where I declare my array.

array[m][n];

Any ideas how to overcome this. I do need such an array as my entire logic depends on how to process this array.

Deepti Jain
  • 1,901
  • 4
  • 21
  • 29

5 Answers5

4

Probably you are running out of stack space.
Can you not allocate the array dynamically on heap using malloc?

You may want to have a look at this answer if you do not know how to do that.

Community
  • 1
  • 1
Alok Save
  • 202,538
  • 53
  • 430
  • 533
  • +1 for a correct answer but the link is no good. No need to go to an array of iterated pointers. Just `double (*array)[n] = malloc(double[m][n]);` would do. – Jens Gustedt Dec 09 '11 at 16:35
2

As others have said it is not a good idea to allocate a large VLA (variable length array) on the stack. Allocate it with malloc:

double (*array)[n] = malloc(sizeof(double[m][n]));

and you have an object as before, that is that the compiler perfectly knows how to address individual elements array[i][j] and the allocation still gives you one consecutive blob in memory.

Just don't forget to do

free(array);

at the end of your scope.

Jens Gustedt
  • 76,821
  • 6
  • 102
  • 177
  • You probably know a lot more about C/memory so please correct me where I'm wrong. With his values of m and n (200, 50000), your answer is asking for one consecutive 300mb chunk of memory whereas my answer is asking for 200 separate 1.5mb chunks. Aren't you much more likely to get the smaller chunks due to memory fragmentation? – Andrew Rasmussen Dec 09 '11 at 17:08
  • I am not sure that understand your question correctly. On modern systems large chunks like this are allocated by 'anonymous mappings', that is the system reserves just an address range in virtual memory and allocates physical pages to it as needed. When deallocating all effects of that allocation are gone. For your method the system may keep the individual chunks around to reuse them later. If you repeat that, you will probably increase the fragmentation a lot. You stress the `malloc` system unnecessarily by doing a lot of allocations that all are then freed at the same time. – Jens Gustedt Dec 09 '11 at 18:44
1

Not sure what type you're using but for the following code I've assumed int.

Rather than doing this:

int array[200][50000];

Try doing this:

int** array = (int**)malloc(200);
for (int i = 0; i < 200; i++)
{
    array[i] = (int*)malloc(50000);
}

This will allocate "heap" memory rather than "stack" memory. You are asking for over 300mb (if you're using a 32bit type) so you probably don't have that much "stack" memory.

Make sure to cleanup after you're done with the array with:

for (int i = 0; i < 200; i++)
{
    free(array[i]);
}
free(array);

Feel free to use m and n instead of the constants I used above!

Edit: I originally wrote this in C++, and converted to C. I am a little more rusty with C memory allocation/deallocation, but I believe I got it right.

Andrew Rasmussen
  • 14,912
  • 10
  • 45
  • 81
0

You are likely running out of stack space.

Windows for instance gives each thread 1MB stack. Assuming the array contains integers and you are creating it on the stack you are creating a 40MB stack variable.

You should instead dynamically allocate it on the heap.

Dave Rager
  • 8,002
  • 3
  • 33
  • 52
0

The array (if local) is allocated in the stack. There is certain limits imposed on the stack size for a process/thread. If the stack is overgrown it will cause issues.

But you can allocate the array in heap using malloc . Typical heap size could be 4GB (this can be more or less depending on OS/Architecture). Check the return value of malloc to make sure that memory for the array is correctly allocated.

Lunar Mushrooms
  • 8,358
  • 18
  • 66
  • 88