Segmentation failure while compiling in gcc in Linux with 128 gb ram

Question

I am running a c code that requires a matrix of size 2000*2000. I have used both malloc and direct allocation. But while compilation it shows "segmentation failure(core-dumped)". I am running it on the terminal of Linux system with 128 gb ram

The first code is below

#include<stdio.h>
int main(int argc,char ** argv){
  int N = 2000;
  int A[N][N];
}

The second one is

#include<stdio.h>
int main(int argc,char ** argv){
int i,j;
  int Nx = 2000;
  int Ny = 2000;
int npoint = Nx*Ny;
 double** A =(double**) malloc(npoint*sizeof(double));
for (j = 0;j< npoint;j =j+1){
    A[j] = (double*)malloc((npoint)*sizeof(double*));
    }
 for( j= 0;j<npoint;j++){
   for (i=0;i<npoint;i++){
    A[i][j]  = 0;
   }}
}

Using gcc to compile this the first one directly shows segmentation failure(core dumped) but the other one starts but after some time gets terminated.

Unrelated: `Void main(int argc,char ** argv)` - what is `Void`? Why not make that `int` instead? — Ted Lyngmo, Jul 24 '23 at 05:55
I'm sure the problem is the same, but what is `Void` and why don't you use `int` instead? — Ted Lyngmo, Jul 24 '23 at 05:57
Seem to have confused allocating pointers vs. allocating doubles.... Examine the `sizeof()` parameters — Fe2O3, Jul 24 '23 at 05:57
Local variables are typically located on the process stack. The process stack is a limited resource, on Windows it defaults to only one single MiB, on Linux 8 MiB. Your array is **16** MiB. That's way too big to fit on the stack. — Some programmer dude, Jul 24 '23 at 05:59
The first example tries to put ~16mb on the stack. Unfortunately the stack is usually not so large. — Retired Ninja, Jul 24 '23 at 05:59
Also note that your second example try to allocate around 1,024×10¹⁵ bytes (assuming 64-bit pointers). That's is way too much. — Some programmer dude, Jul 24 '23 at 06:01
Your second example also allocates in the wrong order. It should *first* allocate an array of pointers, and *then* a set of arrays of `double`. Putting this and the previous comment together you should allocate an array `Nx` pointers to `double`. Then for each element in that array, allocate an array of `Ny` `double` values. — Some programmer dude, Jul 24 '23 at 06:02
Yes, by dynamic allocation. ***Correct*** dynamic allocation. — Some programmer dude, Jul 24 '23 at 06:05
When switching from `A[]` to `A[][]`, the 4 x 10^6 elements become addressed as if they are 16 x 10^12... Code obviously needs less guesswork and more attention to detail. — Fe2O3, Jul 24 '23 at 06:05
Side issue: It is extremely unlikely that you get a segmentation error or a dump while compiling, or linking, or any part of building. — Yunnosch, Jul 24 '23 at 06:09
For the second code to allocate a `double[Ny][Nx]` (instead of what looks like a `double[Ny*Nx][Ny*Nx]`) you could simplify it to `double(*A)[Nx] = calloc(Ny, sizeof *A);` — Ted Lyngmo, Jul 24 '23 at 06:29
I am not able to reproduce the segfault for the first case, but concur with @Someprogrammerdude as to root cause. This is why VLAs are risky especially when they use dynamic values as you have no resource. You can change the stack size under Linux with `ulimit -s`. — Allan Wind, Jul 24 '23 at 06:31
In your 2nd program you are missing an `#include `. It's poorly formatted which makes your code harder to read. — Allan Wind, Jul 24 '23 at 06:34
If you are trying to allocate a 2k x 2k array of doubles initialized to 0 what about `double (*A)[N][N] = calloc(1, sizeof *A);`? In your original code you want to swap the two loops in the initializer so `A[i][j]` to match memory layout, otherwise cache miss will slow it down. — Allan Wind, Jul 24 '23 at 06:49
@AllanWind If you do `double (*A)[N][N] = calloc(1, sizeof *A);` instead of what I suggested, `double(*A)[Nx] = calloc(Ny, sizeof *A);`, then you'd also need `(*A)[i][j]` instead of `A[i][j]`. — Ted Lyngmo, Jul 24 '23 at 07:12
@TedLyngmo Yeah, it was probably not super clear that the 2nd part was a patch against original code. Almost added a @ for you as it was similar when I went back through the comments to ensure it wasn't a duplicate. — Allan Wind, Jul 24 '23 at 07:33

axuwon · Answer 1 · 2023-07-25T02:37:38.997

0

So first of all, you should allocate memory for pointer to array of Nx pointers of sizeof(double*). Next step for each pointer within array you should allocate memory of size Ny * sizeof(double). Below is the code for it.

#include<stdlib.h>

int main(int argc, char ** argv)
{
    int i,j;
    int Nx = 2000;
    int Ny = 2000;
    double** A = malloc(Nx * sizeof(double*));
    for (j = 0; j < Nx; j = j+1){
        A[j] = malloc(Ny * sizeof(double));
    }
    for(j = 0; j < Nx; j++)
        for (i=0; i < Ny; i++)
            A[i][j]  = 0;
}

edited Jul 25 '23 at 02:37

answered Jul 24 '23 at 07:28

axuwon

1
1

2

`A[j]` is out of bounds. You allocate `2000` `double *`, but `npoint` is `2000*2000`, This is part of the problem with original code. I.e. replace `npoint` with `Nx` in the loop and remove the `npoint` variable. – Allan Wind Jul 24 '23 at 07:34
1

It doesn't matter much here but if the array is larger, say, `Nx = Ny = 20000` then the init double loop is 2.8x times slower than if you swap the last two for statements. Also, make `i` and `j` loop local variables. And remove the cast of malloc return value. – Allan Wind Jul 24 '23 at 07:39
Hi, @AllanWind. My bed I've overlooked it. So I've corrected the code snippet. – axuwon Jul 24 '23 at 14:37

Segmentation failure while compiling in gcc in Linux with 128 gb ram

1 Answers1