C assembler function casting

Question

I came across this piece of code (for the whole program see this page, see the program named "srop.c").

My question is regarding how func is used in the main method. I have only kept the code which I thought could be related.

It is the line *ret = (int)func +4; that confuses me.

There are three questions I have regarding this:

1. func(void) is a function, should it not be called with func() (note the brackets)
2. Accepting that that might be some to me unknown way of calling a function, how can it be casted to an int when it should return void?
3. I understand that the author doesn't want to save the frame pointer nor update it (the prologue), as his comment indicates. How is this skipping-two-lines ahead achieved with casting the function to an int and adding four?

.

(gdb) disassemble func
Dump of assembler code for function func:
0x000000000040069b <+0>:     push   %rbp
0x000000000040069c <+1>:     mov    %rsp,%rbp
0x000000000040069f <+4>:     mov    $0xf,%rax
0x00000000004006a6 <+11>:    retq
0x00000000004006a7 <+12>:    pop    %rbp
0x00000000004006a8 <+13>:    retq
End of assembler dump.

Possibly relevant is that when compiled gcc tells me the following:
warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]

Please see below for the code.

void func(void)
{
    asm("mov    $0xf,%rax\n\t");
    asm("retq\n\t");
}

int main(void)
{
    unsigned long *ret;

    /*...*/

    /* overflowing */
    ret = (unsigned long *)&ret + 2;
    *ret = (int)func +4; //skip gadget's function prologue

    /*...*/

    return 0;
}

[Edit] Following the very helpful advice, here are some further information:

    calling func returns a pointer to the start of the function: 0x400530
    casting this to an int is dangerous (in hex) 400530
    casting this to an int in decimal  4195632
    safe cast to unsigned long 4195632

    size of void pointer:  8
    size of int:  4
    size of unsigned long:  8

[Edit 2:] @cmaster: Could you please point me to some more information regarding how to put the assembler function in a separate file and link to it? The original program will not compile because it doesn’t know what the function prog (when put in the assembler file) is, so it must be added either before or during compilation?

Additionally, the gcc -S when ran on a C file only including the assembly commands seem to add a lot of extra information, could not func(void) be represented by the following assembler code?

func:
mov    $0xf,%rax
retq

Answer 1

This code assumes a lot more than what is good for it. Anyway, the snippet that you have shown only tries to produce a pointer to the assembler function body, it does not attempt to call it. Here is what it does, and what it assumes:

• func by itself produces a pointer to the function.

  Assumption 1:
  The pointer actually points to the start of the assembler code for func. That assumption is not necessarily right, there are architectures where a function pointer is actually a pointer to a pair of pointers, one of which points to the code, the other of which points to a data segment.

• func + 4 increments this pointer to point to the first instruction of the body of the function.

  Assumption 2:
  Function pointers can be incremented, and their increment is in terms of bytes. I believe that this is not covered by the C standard, but I may be wrong on that one.

  Assumption 3: The prolog that is inserted by the compiler is precisely four bytes long. There is absolutely nothing that dictates what kind of prolog the compiler should emit, there is a multitude of variants allowed, with very different lengths. The code you've given tries to control the length of the prolog by not passing/returning any parameters, but still there can be compilers that produce a different prolog. Worse, the size of the prolog may depend on the optimization level.

• The resulting pointer is cast to an int.

  Assumption 4:
  sizeof(void (*)(void)) == sizeof(int). This is false on most 64 bit systems: on these systems int is usually still four bytes while a pointer occupies eight bytes. On such a system, the pointer value will be truncated. When the int is cast back into a function pointer and called, this will likely crash the program.

---

My advice:
If you really want to program in assembler, compile a file with only an empty function with gcc -S. This will give you an assembler source file with all the cruft that's needed for the assembler to produce a valid object file and show you where you can add the code for your own function. Modify that file in any way you like, and then compile it together with some calling C code as normal. That way you avoid all these dangerous little assumptions.

Answer 2

1. The name of a function is a pointer to the start of the function. So the author is not calling the function at that point. Just saving a reference to the start of it.

2. It's not a void. It's a function pointer. More precisely in this case it is of type: void (*)(void). A pointer is just an address so can be cast to an int (but the address may be truncated if compiled for a 64 bit system as ints are 32 bits in that case).

3. The first instruction of the function pushes the fp onto the stack. By adding 4, that instruction is skipped. Note that in the snippets you gave the function has not been called. It's probably part of the code that you have not included.