Why can I only assign less than 10 mb using variable-lenght array in C++?

Question

I know that variable-length arrays (VLAs) are technically not allowed in C++. However, unless you use the -pedantic keyword you do not get a warning. And even then, you only get a warning.

Though I have not found a specific reference on this, I am pretty sure VLAs are allocated on the stack while dynamic arrays are allocated on the heap.

When debugging a function which was receiving messages of usually less than 100 mb I ran into a case when the middle of the array could not be accessed using gdb, while both the beginning and the end could. I realize that when allocated on the stack I might run into memory or address space limits quicker.

Why does this code segfault at so low numbers of bytes? Are there any limits to the size of VLAs? And why does the segfault happen on access, not allocation? And why can I access the end of the array using gdb (in this example code, in the other larger program I could access the start as well)?

I get the same result with clang and gcc.

# include <iostream>
# include <vector>

using std::cout;
using std::endl;


void foo_a (int n) {
  /* on stack */
  cout << "a: (C),   n = " << n << endl;
  char buffer[n]; buffer[n] = '\0';

  cout << (void*)buffer << endl;

  for (int i = 0; i < n; i++) {
    buffer[i] = (char) i;
  }
}

void foo_b (int n) {
  /* on heap */
  cout << "b: (C++), n = " << n << endl;
  char * buffer = new char[n];

  for (int i = 0; i < n; i++) {
    buffer[i] = (char) i;
  }

  cout << (void*)buffer << endl;
  delete [] buffer;
}

int main (int, char**) {
  int Ns[] = { 1024, 123123, 10586239 };

  for (int n : Ns) {
    foo_b (n);
    foo_a (n);
  }

  return 0;
}

Remember each thread has to have a stack. So if stacks were GB that would limit the # of threads you could use in your program. A heap is shared by all threads. — drescherjm, Oct 20 '18 at 18:34
@gauteh Probably depends on the compiler specific implementation. It's hard to tell you something resilient about this. — πάντα ῥεῖ, Oct 20 '18 at 18:35
"*unless you use the -pedantic keyword you do not get a warning*" - `-pedantic` is not a keyword, it's a compiler option; specifically, a GCC option. (Clang also supports it because it mimics gcc's interface.) It's not a general C++ thing. — melpomene, Oct 20 '18 at 18:38
Linux usually never throws on allocation, but on access when it gets out of resources. — Ripi2, Oct 20 '18 at 18:40
[Why aren't variable-length arrays part of the C++ standard?](https://stackoverflow.com/questions/1887097/why-arent-variable-length-arrays-part-of-the-c-standard) — Swordfish, Oct 20 '18 at 18:41

score 2 · Accepted Answer · answered Oct 20 '18 at 18:44

2

Stacks are generally a fixed size. As this size is reserved on thread creation if the size was large then you wouldn't be able to create very many threads before running out of memory.

The stack is designed for small allocations of temporary data, long lived data and large allocations should be performed on the heap.

Typically stack size is around 8mb on Linux and 1mb on Windows.

You may be able to allocate more than the available stack size and your program will only crash when trying to access outside of the available stack size. This depends on the platform, for example on Windows you will get a stack overflow exception as soon as you allocate more than will fit in the stack.

answered Oct 20 '18 at 18:44

Alan Birtles

32,622
4
31
60

1

also on windows - system have reserved and allocated stack size. by default usually reserved near 1Mb, when really allocated only several Kb. when we allocate big space (more that 4kb) in stack compiler (msvc) insert stack "probe" code - page by page with step 4Kb. because all this memory yet not allocated - cpu exception generated on every probe. but kernel view that this is from reserved stack space - handle exception and allocate additional page. only when last page in stack assigned - it allocated too and stack overflow exception passed to user mode. – RbMm Oct 20 '18 at 18:58
from this follows that big stack allocations much more slow compare allocation from heap. – RbMm Oct 20 '18 at 18:58
Why does it sometimes work to allocate up to 20 mb on the stack in a buffer like this, and assign and read all of it? `ulimit -s` gives `8192`. – gauteh Oct 21 '18 at 10:52
I guess the os is just missing the fact you've overflowed the stack – Alan Birtles Oct 21 '18 at 12:56
@AlanBirtles art thou joking with me? the data is not corrupt. – gauteh Oct 22 '18 at 12:15
@gauteh I'm not joking, the OS will only generate a seg fault when you try to access a page in a way that you are not allowed to. If you are reading from the stack and your reads overflow the pages allocated to your stack but happen to go into pages that you are allowed to read you won't get any seg faults – Alan Birtles Oct 22 '18 at 12:26

Why can I only assign less than 10 mb using variable-lenght array in C++?

1 Answers1