Problem with call windows API inline assembly

Question

I want to anti-debug and write a function like following code to call API debug "IsDebuggerPresent" to check:

#include "windows.h"
bool checkdbg(){
    int i = 1;
    __asm{
        call IsDebuggerPresent //gọi api debug
        test eax, eax
        jne L1
        mov i,0
        L1 : 
    }
    if(i == 0) return false;
    else return true;
}

But when compile,VS2010 can not find IsDebuggerPresent to call.Please help me !

Answer 1

#include "Windows.h"

bool checkdbg() {
    __asm {
        call IsDebuggerPresent //gọi api debug
        test eax, eax
        jne L1
        mov i,0
        L1 : 
    }
    if(i == 0) return false;
    else return true;
}

int CALLBACK WinMain( HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow ) {
  checkdbg();
  return ERROR_SUCCESS;
}

This will compile just fine. It's probably complaining because you were missing an entry point. The subsystem you supplied in the project options mandates what main is required, whether that be WinMain (for Windows subsystem) or main (for console subsystem).

Also I would advise using:

#include <Windows.h>

As opposed to:

#include "Windows.h"

See here for explanation. Also you should just do this:

#include <Windows.h>

BOOL checkdbg() {
    int i = 1;
    __asm{
        call IsDebuggerPresent
        ret
    }
}

int main() {
  checkdbg();
  return ERROR_SUCCESS;
}

The return is expected in eax anyway so you may as well return, otherwise all you're doing is checking if the return is true, then return true. If it's false, return false. Just return. The only thing you did was effectively cast from BOOL to bool. Beats me why you're using inline ASM for this anyway though.

Answer 2

do not use

call balabala

use

call dword ptr [balabala]

Answer 3

any reason for not calling the API directly from C ?

#include <windows.h>
#pragma comment(lib, "kernel32.lib")

bool checkdbg() {
    return IsDebuggerPresent();
}