2

Problem

I'm trying to declare a large 2D Array (a.k.a. matrix) in C / C++, but it's crashing with segfault only on Linux. The Linux system has much more RAM installed than the macOS laptop, yet it only crashes on the Linux system.

My question is: Why does this crash only on Linux, but not macOS?

Here is a small program to reproduce the issue:

// C++ program to segfault on linux
#include <iostream>
#include <stdlib.h>

using namespace std;

int main()
{
    cout << "Let's crash for no raisin! " << endl;

    cout << "Int size: " << sizeof(int) << endl;
    for (int n=1012; n < 2000; n++) {
      cout << "Declaring Matrix2D of size: " << n << "x" << n << " = " << n*n << endl;
      cout << "Bytes: " << n*n*sizeof(int) << endl;

      // segfault on my machine at 1448x1448 = 8386816 bytes
      int Matrix2D[n][n];
      // these two lines can be commented out and the program still reaches segfault
      // int* pM2D = (int*)malloc(n*n*sizeof(int));
      // free(pM2D);
    }
    return 0;
}

Compile with: g++ -Wall -g -o segfault segfault.cpp

Output

Linux

Linux system has 64 GiB RAM installed!

$ ./segfault ; free --bytes
Let's crash for no raisin! 
Int size: 4

[...SNIP...]

Declaring Matrix2D of size: 1446x1446 = 2090916
Bytes: 8363664
Declaring Matrix2D of size: 1447x1447 = 2093809
Bytes: 8375236
Declaring Matrix2D of size: 1448x1448 = 2096704
Bytes: 8386816
Segmentation fault (core dumped)


              total        used        free      shared  buff/cache   available
Mem:    67400994816 11200716800  4125982720   412532736 52074295296 55054041088
Swap:    1023406080   824442880   198963200

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 256763
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 256763
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
macOS

macOS system has only 16 GB RAM installed! 

$ ./segfault ; sysctl -a | grep mem ;
Let's crash for no raisin! 
Int size: 4

[...SNIP...]

Declaring Matrix2D of size: 1997x1997 = 3988009
Bytes: 15952036
Declaring Matrix2D of size: 1998x1998 = 3992004
Bytes: 15968016
Declaring Matrix2D of size: 1999x1999 = 3996001
Bytes: 15984004


kern.dtrace.buffer_memory_maxsize: 5726623061
kern.dtrace.buffer_memory_inuse: 0
kern.memorystatus_sysprocs_idle_delay_time: 10
kern.memorystatus_apps_idle_delay_time: 10
kern.memorystatus_purge_on_warning: 2
kern.memorystatus_purge_on_urgent: 5
kern.memorystatus_purge_on_critical: 8
vm.memory_pressure: 0
hw.memsize: 17179869184
machdep.memmap.Conventional: 17077571584
machdep.memmap.RuntimeServices: 524288
machdep.memmap.ACPIReclaim: 188416
machdep.memmap.ACPINVS: 294912
machdep.memmap.PalCode: 0
machdep.memmap.Reserved: 84250624
machdep.memmap.Unusable: 0
machdep.memmap.Other: 0

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 65532
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2784
virtual memory          (kbytes, -v) unlimited
TrinitronX
  • 4,959
  • 3
  • 39
  • 66
  • 3
    In Standard C++, the size of an array must be a compile time constant. You can/should use `std::vector` instead. – Jason Jan 11 '22 at 08:08
  • 2
    It's probably a matter of default stack size which might me diffferent on Linux and on MacOS., local variables being usually stored on the stack. – Jabberwocky Jan 11 '22 at 08:12
  • As Matrix2D is never used, the compiler is free to optimize it away. And Annop Rana is right, this is not the legit way to create an array. – SKCoder Jan 11 '22 at 08:12
  • 3
    Anyway `int Matrix2D[n][n];` for creating an dynamically sized array is not legal C++, it's available as an extension on some compilers. I'd rather use `std::vector`. – Jabberwocky Jan 11 '22 at 08:14
  • 2
    If you are using clang on macos and gcc on linux, a quick test on godbolt seems to indicate that (at least without extra flags), gcc probes pages when doing the large allocation, so that the segfault will be triggered once the stack limit is exceeded. Clang doesn't seem to be doing that, so it never accesses the memory and thus no segfault. Try writing to some random position in the array and see whether you get your segfault. – user17732522 Jan 11 '22 at 08:24
  • Either switch to `std::vector< std::vector > Matrix2D;` or `int Matrix2D[1447][1447];` so you don't blow up the stack. – digito_evo Jan 11 '22 at 08:24
  • 1
    `int Matrix2D[n][n];` --> `std::vector> Matrix2D(n, std::vector(n));` -- Then there would be no issues, plus the code would be valid C++ that would work across compilers. – PaulMcKenzie Jan 11 '22 at 08:55
  • @user17732522 That's really interesting! On second thought, I set `ulimit -s 8192` on _macOS_ and still could not trigger the segfault. Perhaps it does come down to compiler differences after all – TrinitronX Jan 11 '22 at 09:11

1 Answers1

5

Although ISO C++ does not support variable-length arrays, you seem to be using a compiler which supports them as an extension.

In the line

int Matrix2D[n][n];

n can have a value up to 2000. This means that the 2D array can have 2000*2000 elements, which equals 4 million. Every element has a size of sizeof(int), which is 4 bytes on linux. This means that you are allocating a total of 16 Megabytes on the stack. This is exceeding the limit of the stack, causing a stack overflow.

The reason why it is not crashing on MacOS could be that the stack is configured for a higher maximum limit, or it could be that your program is not crashing because variable-length arrays are implemented differently, so that the program is not touching the 2D array, or maybe it is touching the 2D array, but only in such a way that it doesn't cause a crash. These are implementation-details of the compiler.

The amount of memory actually installed on the computer is not relevant. What counts is the maximum stack limit configured in the operating system.

If you want to use larger amounts of memory than would be permitted on the stack, you should use the heap instead. In that case, you should allocate the memory instead with std::make_unique, operator new or std::malloc. You can also use most STL containers such as std::vector, which will automatically store its contents on the heap, even if you create the actual container on the stack. However, beware that some STL containers will not, such as std::array.

Andreas Wenzel
  • 22,760
  • 4
  • 24
  • 39
  • I suppose the next question is why does it work on macOS? Is the stack limit higher on that operating system? – TrinitronX Jan 11 '22 at 08:21
  • 1
    @TrinitronX this might be interesting: https://stackoverflow.com/questions/42315207/extend-the-stack-size-on-macos-sierra – Jabberwocky Jan 11 '22 at 08:23
  • @Jabberwocky Thanks! That uncovered the differences between settings on each OS: _macOS_ `ulimit -Hs = 65532`, _Linux_ `ulimit -Hs = unlimited`. While Linux has no high limit, the default setting I found was lower: _macOS_ `ulimit -s = 65532`, _Linux_ `ulimit -s = 8192` – TrinitronX Jan 11 '22 at 08:28
  • @Jabberwocky As it turns out, matching `ulimit -s 65532` on _Linux_ now gives me the same result as on _macOS_. Solved the mystery! – TrinitronX Jan 11 '22 at 08:30
  • Those are bad choices in most cases. Why not use a unique_ptr to handle the buffer instead of using dangerous `malloc` or `new`? – digito_evo Jan 11 '22 at 08:33
  • 1
    @digito_evo: Yes, you are right. I have now added `make_unique` as the first item in the list. – Andreas Wenzel Jan 11 '22 at 08:38
  • @AndreasWenzel IMO the first choice should have been `std::vector`, and `malloc` would be the last, if there at all. – PaulMcKenzie Jan 11 '22 at 08:52
  • Yes the macOS stack is 65 MB, and each macho64 thread gets its own. – Richard Barber Jan 11 '22 at 09:05