3

Does Linux provide an inaccessible memory area below the lower stack end that has a guaranteed minimum size? And if such a guaranteed minimum size exists, what is it?

Or in other words, when should I start to worry about alloca() or so giving me pointers into valid, non-stack memory?

thejh
  • 44,854
  • 16
  • 96
  • 107
  • Does any system provide that? – Elliott Frisch Mar 20 '14 at 17:41
  • @ElliottFrisch Well, I hope so... otherwise you could cause the stack to run into the heap for any program that can be forced to allocate enough RAM, then to perform arbitrarily large recursion. – thejh Mar 20 '14 at 17:43
  • Okay. What is it called on the systems where you have seen it? [Stack smashing protection?](http://wiki.osdev.org/GCC_Stack_Smashing_Protector) – Elliott Frisch Mar 20 '14 at 17:44
  • @ElliottFrisch No, that's a protection against buffer overflows, not "underflows" of the stack, and it happens in the compiler/in userspace and is not enforced using a memory mapping or so. – thejh Mar 21 '14 at 05:54
  • 1
    This question is more or less a duplicate of http://stackoverflow.com/questions/5543330/how-many-memory-pages-do-c-compilers-on-desktop-oses-use-to-detect-stack-overflo – Pascal Cuoq Mar 24 '14 at 17:57

2 Answers2

4

As the alloca man page says:

There is no error indication if the stack frame cannot be extended. (However, after a failed allocation, the program is likely to receive a SIGSEGV signal if it attempts to access the unallocated space.)

So there is no indication at all and it also says:

If the allocation causes stack overflow, program behavior is undefined.

The stack overflow problem is a general issue with recursion and not really particular to alloca or let's say variable length arrays. Typically you either need to find a way to limit the depth of the recursion, refactor to an iterative solution or use your own dynamic stack(probably does not apply to this case).

Update

As the OP discovered Linux does provide an after the fact indication using a guard page after the stack of stack overflow by generating a SIGBUS signal, which addresses the first part of the question.

Shafik Yaghmour
  • 154,301
  • 39
  • 440
  • 740
  • 1
    "As the OP discovered Linux does provide an after the fact indication of stack overflow by generating a SIGBUS signal." But my question is about the reliability of that for large allocations – you don't always get a signal for this, sometimes your program silently goes on and does idiotic things. My question is what the size maximum for alloca() is before I have to worry about my program doing really bad things instead of just crashing. – thejh Mar 21 '14 at 05:52
  • @thejh I added that just for completeness since your question really had two parts. This problem is more general to recursion and the solution requires a redesign. – Shafik Yaghmour Mar 21 '14 at 13:09
  • 1
    Sure, when you mess up your code so that there are code paths with unbounded recursion, that should be fixed – but it's a huge difference whether someone can crash my program or can make it run arbitrary code, and that's what my question was about. – thejh Mar 22 '14 at 13:07
1

Thanks to @ElliottFrisch for making me google this with the proper name... whoops.

Looks like the answer is "in newer kernels: one page, in older kernels: no such protection".

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=320b2b8de12698082609ebbc1a17165727f4c893

thejh
  • 44,854
  • 16
  • 96
  • 107