Is there a way to get the return value of a function that is in the shellcode, without using pointer to function?
#include <stdio.h>
unsigned char code[] = "\x55\x48\x89\xe5"
"\xb8\x05\x00\x00"
"\x00\x5d\xc3";
int main(void) {
int (*p)(void) = (int(*)(void))code;
printf("%d", p());
return 0;
}
Shellcode (see Wikipedia article Shellcode as well as this presentation Introduction to Shellcode Development) is machine code that is injected into an application in order to take over the application and run your own application within that application's process.
How the shellcode is injected into the application and starts running will vary depending on how the penetration is being done.
However for testing approaches for the actual shellcode, as opposed to approaches for injecting the shellcode in the first place, the testing is typically done with a simple program that allows you to (1) create the shellcode program that is to be injected as an array of bytes and (2) start the shellcode executing.
The simplest approach for this is the source code you have posted.
You have an array of unsigned char which contains the machine code to be executed.
You have a main() which creates a function pointer to the array of unsigned char bytes and then calls the shellcode through the function pointer.
However in a real world penetration what you would normally do is to use a technique whereby you would take over an application by injecting your shellcode into its process space and then triggering the execution of that shellcode. One such approach is a buffer overflow attack. See for example COEN 152 Computer Forensics Buffer Overflow Attack as well as Wikipedia article Buffer overflow.
See also
Also note that the approaches for shellcode attacks will vary depending on the operating system that is being attacked. For instance see this article Basics of Windows shellcode writing which explains some of the intricacies of writing a shellcode for accessing system calls in Windows. Compare to this article providing a way for How to write a Linux x86 shellcode.
