How to intercept a static library call in C language?

Question

Here's my question: There is a static library (xxx.lib) and some C files who are calling function foo() in xxx.lib. I'm hoping to get a notification message every time foo() is called. But I'm not allowed to change any source code written by others.

I've spent several days searching on the Internet and found several similar Q&As but none of these suggestions could really solve my problem. I list some of them:

1. use gcc -wrap: Override a function call in C Thank god, I'm using Microsoft C compiler and linker, and I can't find an equivalent option as -wrap.

2. Microsoft Detours: Detours intercepts C calls in runtime and re-direct the call to a trampoline function. But Detours is only free for IA32 version, and it's not open source.

3. I'm thinking about injecting a jmp instruction at the start of function foo() to redirect it to my own function. However it's not feasible when foo()is empty, like

     void foo() --->  will be compiled into 0xC3 (ret)
     {                but it'll need at least 8 bytes to inject a jmp
     }
   

4. I found a technology named Hotpatch on MSDN. It says the linker will add serveral bytes of padding at the beginning of each function. That's great, because I can replace the padding bytes with jmp instruction to realize the interception in runtime! But when I use the /FUNCTIONPADMIN option with the linker, it gives me a warning:

   LINK : warning LNK4044: unrecognized option '/FUNCTIONPADMIN'; ignored
   

   Anybody could tell me how could I make a "hotpatchable" image correctly? Is it a workable solution for my question ?

Do I still have any hope to realize it ?

Answer 1

If you have the source, you can instrument the code with GCC without changing the source by adding -finstrument-functions for the build of the files containing the functions you are interested in. You'll then have to write __cyg_profile_func_enter/exit functions to print your tracing. An example from here:

#include <stdio.h>
#include <time.h>

static FILE *fp_trace;

void
__attribute__ ((constructor))
trace_begin (void)
{
 fp_trace = fopen("trace.out", "w");
}

void
__attribute__ ((destructor))
trace_end (void)
{
 if(fp_trace != NULL) {
 fclose(fp_trace);
 }
}

void
__cyg_profile_func_enter (void *func,  void *caller)
{
 if(fp_trace != NULL) {
 fprintf(fp_trace, "e %p %p %lu\n", func, caller, time(NULL) );
 }
}

void
__cyg_profile_func_exit (void *func, void *caller)
{
 if(fp_trace != NULL) {
 fprintf(fp_trace, "x %p %p %lu\n", func, caller, time(NULL));
 }
}

Another way to go if you have source to recompile the library as a shared library. From there it is possible to do runtime insertions of your own .so/.dll using any number of debugging systems. (ltrace on unix, something or other on windows [somebody on windows -- please edit]).

If you don't have source, then I would think your option 3 should still work. Folks writing viruses have been doing it for years. You may have to do some manual inspection (because x86 instructions aren't all the same length), but the trick is to pull out a full instruction and replace it with a jump to somewhere safe. Do what you have to do, get the registers back into the same state as if the instruction you removed had run, then jump to just after the jump instruction you inserted.

Answer 2

The VC compiler provides 2 options /Gh & /GH for hooking functions.

The /Gh flag causes a call to the _penter function at the start of every method or function, and the /GH flag causes a call to the _pexit function at the end of every method or function.

So, if I write some code in _penter to find out the address of the caller function, then I'll be able to intercept any function selectively by comparing the function address.

I made a sample:

#include <stdio.h>

void foo()
{

}

void bar()
{

}

void main() {
  bar();
  foo();
  printf ("I'm main()!");
}


void __declspec(naked) _cdecl _penter( void ) 
{
    __asm {
        push ebp;               // standard prolog
        mov ebp, esp;
        sub esp, __LOCAL_SIZE
        pushad;                 // save registers
    }

    unsigned int addr;
    // _ReturnAddress always returns the address directly after the call, but that is not the start of the function!
    // subtract 5 bytes as instruction for call _penter
    // is 5 bytes long on 32-bit machines, e.g. E8 <00 00 00 00>
    addr = (unsigned int)_ReturnAddress() - 5;

    if (addr == foo) printf ("foo() is called.\n");
    if (addr == bar) printf ("bar() is called.\n");

    _asm {
        popad;              // restore regs
        mov esp, ebp;       // standard epilog
        pop ebp;
        ret;
    }
}

Build it with cl.exe source.c /Gh and run it:

bar() is called. foo() is called. I'm main()!

It's perfect!

More examples about how to use _penter and _pexit can be found here A Simple Profiler and tracing with penter pexit and A Simple C++ Profiler on x64.

I've solved my problem using this method, and I hope it can help you also.

:)

Answer 3

I don't think there is any to do this without changing any code. Easiest way I can think of is to do this is to write wrapper for your void foo() function and Find/Replace it with your wrapper.

void myFoo(){
     return foo();
}

Instead of calling foo() call myFoo().

Hope this will help you.