Shell code is the payload used when exploiting a vulnerability that is used to create a command shell from which the attacker can control the machine.
A typical shell code when run might open a network connection and spawn cmd.exe
on a Windows machine (or /bin/sh
on Linux/unix) piping stdin
and stdout
over the network connection. An attacker may complete the connection from his machine and enter commands and get feedback as if he was sitting at the compromised machine.
A buffer overflow is not shell code. It is the vulnerability that is exploited to execute the shell code.
The buffer overflow is exploited to copy the shell code to the user's machine and overwrite the return address on the program's stack. When the currently executing function returns, the processor jumps to the uploaded shell code which creates the shell for the attacker.
For more information on exploiting buffer overflows, have a look at Smashing the Stack for Fun and Profit.
You can try to use the -fno-stack-protector
flag for gcc
but I'm not very familiar with OSX or whatever stack protections it may use.
If you want to play around with buffer overflows, modern compilers and modern OSs have protections in place that make this difficult. Your best bet would be to grab yourself a Linux distro and turn them off. See this question for more information on disabling these protections.
Note you don't need to have a buffer overflow to execute a shell code. I've demonstrated opening a remote shell using a command injection exploit to upload and execute a batch file.