5

I've made a pin tool to dump CreatFile win32 calls (in my case CreateFileW) and its return values. It looks like this:

/* ... */

VOID Image(IMG img, VOID *v)
{
    RTN cfwRtn = RTN_FindByName(img, "CreateFileW");
    if (RTN_Valid(cfwRtn))
    {
        RTN_Open(cfwRtn);
    
        RTN_InsertCall(cfwRtn, IPOINT_BEFORE, (AFUNPTR)CreateFileWArg,
        IARG_ADDRINT, "CreateFileW",
        IARG_FUNCARG_ENTRYPOINT_VALUE, 0,
        IARG_END);
        RTN_InsertCall(cfwRtn, IPOINT_AFTER, (AFUNPTR)CreateFileWafter,
        IARG_FUNCRET_EXITPOINT_VALUE, IARG_END);

        RTN_Close(cfwRtn);
    }
}

/* ... */

VOID CreateFileWArg(CHAR * name, wchar_t * filename)
{
    TraceFile << name << "(" << filename << ")" << endl;
}

VOID CreateFileWafter(ADDRINT ret)
{
    TraceFile << "\tReturned handle: " << ret << endl;
}

It gives interesting results. For instance, on a small program that just opens an existing file and does nothing else, it gives:

CreateFileW(file.txt)
    Returned handle: 0
CreateFileW(file.txt)
    Returned handle: 0x74
    Returned handle: 0x74

Lots of anomalies.

  1. Why are there two calls?
  2. If i'm not mistaken CreateFile should never ever return 0.
  3. After the second call, it returns twice (?)

I also tried to instrument a simple c++ program, that directly calls CreateFileW once, the result:

CreateFileW(file.txt)
    Returned handle: 0
CreateFileW(file.txt)
    Returned handle: 0xffffffff
    Returned handle: 0xffffffff

The file i tried to open did not exist, so the return value (-1 == INVALID_HANDLE_VALUE) is correct at least.

Any ideas? Thanks in advance!

Heyji
  • 1,113
  • 8
  • 26
Donpedro
  • 828
  • 7
  • 22

2 Answers2

3

Okay, after some time i finally figured out the causes of these problems.

About the return value appearing to be 0:

Well, the PIN documentation says:

NOTE: IPOINT_AFTER is implemented by instrumenting each return instruction in a routine. Pin tries to find all return instructions, but success is not guaranteed

If you dump the function's address at the returns, it turns out that 0 is not returned from CreateFileW. It's returned from another function CreateFileW calls into. This erroneous behaviour of PIN can be fixed with wrapping the CreateFileW method in your own version (dump parameters, call the original function, dump the return value).

About the two function calls instead of only one:

Turns out that on my system, CreateFileW calls into Kernelbase.dll's function, that has exactly the same name. Since i instrumented routines by their name, this is correct behaviour. Checking the image name against kernel32.dll solved this.

Donpedro
  • 828
  • 7
  • 22
  • Thank you for the valuable information about such classic (yet unpredictible) pitfalls one can encounter when trying to instrument such functions. – Heyji May 11 '21 at 05:49
1

I would have suggested to catch it at the system call level, rather than in one of those unsure intermediate level (whichever library it is hosted in). On windows, system call numbers and interface are not officially public, but one can easily find them anyway.

Heyji
  • 1,113
  • 8
  • 26
  • but how can we distinguish between system calls that are made from the main application and the ones that are not? – maysara Mar 14 '21 at 17:51
  • Have look at Nt and Zw functions https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/using-nt-and-zw-versions-of-the-native-system-services-routines – Simone Aonzo Apr 21 '21 at 22:05
  • @Maysara Alhindi: that is a good point. Though it depends on the main application. I would try to filter them by their arguments. You should have hints on system call arguments coming from your main application. If not, it is indeed not the right solution. – Heyji May 11 '21 at 05:52