71

I have enabled the -Wstack-protector warning when compiling the project I'm working on (a commercial multi-platform C++ game engine, compiling on Mac OS X 10.6 with GCC 4.2). This flag warns about functions that will not be protected against stack smashing even though -fstack-protector is enabled. GCC emits some warnings when building the project:

not protecting function: no buffer at least 8 bytes long
not protecting local variables: variable length buffer

For the first warning, I found that it is possible to adjust the minimum size a buffer must have when used in a function, for this function to be protected against stack smashing: --param ssp-buffer-size=X can be used, where X is 8 by default and can be as low as 1.

For the second warning, I can't suppress its occurrences unless I stop using -Wstack-protector.

  1. When should -fstack-protector be used? (as in, for instance, all the time during dev, or just when tracking bugs down?)
  2. When should -fstack-protector-all be used?
  3. What is -Wstack-protector telling me? Is it suggesting that I decrease the buffer minimum size?
  4. If so, are there any downsides to putting the size to 1?
  5. It appears that -Wstack-protector is not the kind of flag you want enabled at all times if you want a warning-free build. Is this right?
General Grievance
  • 4,555
  • 31
  • 31
  • 45
Guillaume
  • 4,901
  • 5
  • 24
  • 19
  • Do you need to use the variable-length arrays (VLA) feature of C99? If you didn't use it, you wouldn't get the warning - and you would get the protection. Are you really sure you gain enough from using VLA that the loss of protection is acceptable? – Jonathan Leffler Oct 28 '09 at 04:50
  • VLAs have been removed, the warning was useful in this case. – Guillaume Oct 29 '09 at 08:26
  • These options try to prevent against buffer overflow/stack-corruption based attacks. These articles should help: - [What's the stack smashing protector](http://www.trl.ibm.com/projects/security/ssp/) - [SSP](http://www.linuxfromscratch.org/hints/downloads/files/ssp.txt) – dirkgently Oct 27 '09 at 09:49
  • Does your company have a security-threat model that you must adhere to? In that case, yes, you'd need that. – dirkgently Oct 29 '09 at 10:01
  • Games Editors expect people to try and crack their games. If I'm not mistaken, these people usually succeed, so why bother putting a protection if there's a performance hit for every player. Stopping the debate right now, I can conclude that using stack-protection on a release build is not for me to decide (as a programmer). I can now bring this discussion to my hierarchy, and indeed do some performance tests when the project is more advanced, so thanks for your answer. :) What about the development stage. How much does stack-protection bring to the table? – Guillaume Oct 29 '09 at 09:53
  • I'd rather do this on debug builds and leave it out on the first release and see how people respond to the game (which IMHO is more important) and keep my fingers crossed that someday, I'll have to put the stack-protection in ;-) – dirkgently Oct 29 '09 at 09:00
  • Do you expect people to try and crack your code? If so, it's probably a good idea. As for performance, you'll have to calculate. Here's a starting point: http://www.trl.ibm.com/projects/security/ssp/node5.html#SECTION00051000000000000000 – dirkgently Oct 29 '09 at 08:58
  • Thanks for the links. I still don't see the big picture though. Is stack protection really useful for a commercial game? Will the game experience any performance hit if -fstack-protector is enabled? And the miminum buffer size set to 1? Or if -fstack-protector-all is enabled? – Guillaume Oct 29 '09 at 08:35

2 Answers2

80

Stack-protection is a hardening strategy, not a debugging strategy. If your game is network-aware or otherwise has data coming from an uncontrolled source, turn it on. If it doesn't have data coming from somewhere uncontrolled, don't turn it on.

Here's how it plays out: If you have a bug and make a buffer change based on something an attacker can control, that attacker can overwrite the return address or similar portions of the stack to cause it to execute their code instead of your code. Stack protection will abort your program if it detects this happening. Your users won't be happy, but they won't be hacked either. This isn't the sort of hacking that is about cheating in the game, it's the sort of hacking that is about someone using a vulnerability in your code to create an exploit that potentially infects your user.

For debugging-oriented solutions, look at things like mudflap.

As to your specific questions:

  1. Use stack protector if you get data from uncontrolled sources. The answer to this is probably yes. So use it. Even if you don't have data from uncontrolled sources, you probably will eventually or already do and don't realize it.

  2. Stack protections for all buffers can be used if you want extra protection in exchange for some performance hit. From gcc4.4.2 manual:

    -fstack-protector

    Emit extra code to check for buffer overflows, such as stack smashing attacks. This is done by adding a guard variable to functions with vulnerable objects. This includes functions that call alloca, and functions with buffers larger than 8 bytes. The guards are initialized when a function is entered and then checked when the function exits. If a guard check fails, an error message is printed and the program exits.

    -fstack-protector-all

    Like -fstack-protector except that all functions are protected.

  3. The warnings tell you what buffers the stack protection can't protect.

  4. It is not necessarily suggesting you decrease your minimum buffer size, and at a size of 0/1, it is the same as stack-protector-all. It is only pointing it out to you so that you can, if you decide redesign the code so that buffer is protected.
  5. No, those warnings don't represent issues, they just point out information to you. Don't use them regularly.
RJFalconer
  • 10,890
  • 5
  • 51
  • 66
brantgurga
  • 1,128
  • 10
  • 14
  • 1
    is there any reason why -fstack-protector only check functions with buffers larger than "8 bytes"? why not 4 bytes or 1 byte? – wenchiching Dec 22 '14 at 11:58
  • 1
    @wenchiching, the reason is performance and likelihood of overflow. 8 bytes was found to be the threshold where the likeliness of overflow was worth the performance cost of the protection. – brantgurga Dec 23 '14 at 12:34
  • 4
    Debugging strategy for stack protection is possible and sometimes useful: (1) execute the binary compiled with stack protection and disable ASLR (address space layout randomization) in a debugger. (2) wait for a core. (3) the core will indicate the address of the stack canary that got clobbered. (4) re-run your binary in the debugger and set a hardware watch point on the address of the clobbered stack canary. (5) the debugger will stop when the memory located at culprit address is modified. – Yeow_Meng Jun 24 '16 at 17:27
  • The mechanism is well described in: https://wiki.osdev.org/Stack_Smashing_Protector – Rachid K. Mar 13 '22 at 16:30
1

You indeed should not care about the warning for normal builds. It's really more of an informational message. I hope it's obvious that you do have an inherent security concern with variable-sized buffers on the stack; get the size calculation wrong and you're opening a big hole.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
MSalters
  • 173,980
  • 10
  • 155
  • 350
  • From: http://osdir.com/ml/cocoa-dev/2009-06/msg01658.html Disable all stack smashing warnings with: -Wno-stack-protector There is no pragma yet to disable only this warning. (This often happens with GCC. There is often a tension between having no warnings or too few warnings. One solution is to make special make target for the offending file and have special compile flags for it.) – Prof. Falken Oct 27 '09 at 11:30
  • Can you elaborate on why variable-size buffers are significantly different than fixed-size buffers in terms of risk? I can see why it is hard to place them in a function as the stack protector does for fixed size buffers. – Jonathan Leffler Oct 28 '09 at 04:48
  • Double attack vectors: an attacker can more provide input bytes than you allocated for the buffer, or he can try to find bugs in your buffer size allocation – MSalters Oct 28 '09 at 08:11