Call SetWindowsHookEx with method defined in header file

Question

I am attempting to add a low level mouse hook to a class. I am able to do so by placing this function in my CPP file:

LRESULT CALLBACK MouseHookProc(int nCode, WPARAM wParam, LPARAM lParam)
{
    //my hook code here
    return CallNextHookEx(0, nCode, wParam, lParam);
} 

Then, I set up the hook in the class constructor like so:

HHOOK mousehook = SetWindowsHookEx(WH_MOUSE_LL, MouseHookProc, NULL, 0);

This works fine for intercepting mouse events, however since the callback function is not defined in my class, it does not have access to any of my class variables.

Therefore, I tried defining the callback function in my header file like so:

LRESULT CALLBACK MouseHookProc(int nCode, WPARAM wParam, LPARAM lParam);

and in my CPP file like this (TMainForm being the class):

LRESULT CALLBACK TMainForm::MouseHookProc(int nCode, WPARAM wParam, LPARAM lParam)
{
     //my hook code here
     return CallNextHookEx(0, nCode, wParam, lParam);
}

However, when I attempt to compile like this, I get the following errors:

[bcc32 Error] MainFormU.cpp(67): E2034 Cannot convert 'long (__stdcall * (_closure )(int,unsigned int,long))(int,unsigned int,long)' to 'long (__stdcall *)(int,unsigned int,long)'

[bcc32 Error] MainFormU.cpp(67): E2342 Type mismatch in parameter 'lpfn' (wanted 'long (__stdcall *)(int,unsigned int,long)', got 'void')

What exactly am I doing wrong here? How is the method now different since I have made it a part of my TMainForm class?

Answer 1

You cannot use a non-static class methods as the callback. Non-static methods have a hidden this parameter, thus the signature of the callback does not match the signature that SetWindowsHookEx() is expecting. Even if the compiler allowed it (which can only be done with casting), the API would not be able to account for the this parameter anyway.

If you want to make the callback be a member of the class (so it can access private fields and such), it has to be declared as static to remove the this parameter, but then you will have to use the form's global pointer to reach it when needed, eg:

class TMainForm : public TForm
{
private:
    HHOOK hMouseHook;
    static LRESULT CALLBACK MouseHookProc(int nCode, WPARAM wParam, LPARAM lParam);
    void MouseHook(int nCode, WPARAM wParam, LPARAM lParam);
public:
    __fastcall TMainForm(TComponent *Owner);
    __fastcall ~TMainForm();
};
extern TMainForm *MainForm;

__fastcall TMainForm::TMainForm(TComponent *Owner)
    : TForm(Owner)
{
    hMouseHook = SetWindowsHookEx(WH_MOUSE_LL, &MouseHookProc, NULL, 0);
}

__fastcall TMainForm::~TMainForm()
{
    if (hMouseHook)
        UnhookWindowsHookEx(hMouseHook);
}

LRESULT CALLBACK TMainForm::MouseHookProc(int nCode, WPARAM wParam, LPARAM lParam)
{
    MainForm->MouseHook(nCode, wParam, lParam);
    return CallNextHookEx(0, nCode, wParam, lParam);
}

void TMainForm::MouseHook(int nCode, WPARAM wParam, LPARAM lParam)
{
    // my hook code here
}

With that said, you should consider using the Raw Input API instead of SetWindowsHookEx(). The LowLevelMouseProc documentation even says so:

Note Debug hooks cannot track this type of low level mouse hooks. If the application must use low level hooks, it should run the hooks on a dedicated thread that passes the work off to a worker thread and then immediately returns. In most cases where the application needs to use low level hooks, it should monitor raw input instead. This is because raw input can asynchronously monitor mouse and keyboard messages that are targeted for other threads more effectively than low level hooks can. For more information on raw input, see Raw Input.

Using Raw Input, the mouse will send WM_INPUT messages directly to your window.

If you are using VCL, you can override the virtual WndProc() method to handle the WM_INPUT message, no static method needed:

class TMainForm : public TForm
{
protected:
    virtual void __fastcall CreateWnd();
    virtual void __fastcall WndProc(TMessage &Message);
};

void __fastcall TMainForm::CreateWnd()
{
    TForm::CreateWnd();

    RAWINPUTDEVICE Device = {0};
    Device.usUsagePage =  0x01;
    Device.usUsage = 0x02;
    Device.dwFlags = RIDEV_INPUTSINK;
    Device.hwndTarget = this->Handle;

    RegisterRawInputDevices(&Device, 1, sizeof(RAWINPUTDEVICE));
}

void __fastcall TMainForm::WndProc(TMessage &Message)
{
    if (Message.Msg == WM_INPUT)
    {
        HRAWINPUT hRawInput = (HRAWINPUT) Message.LParam;
        UINT size = 0;
        if (GetRawInputData(hRawInput, RID_INPUT, NULL, &size, sizeof(RAWINPUTHEADER)) == 0)
        {
            LPBYTE buf = new BYTE[size];

            if (GetRawInputData(hRawInput, RID_INPUT, buf, &size, sizeof(RAWINPUTHEADER)) != 0)
            {
                RAWINPUT *input = (RAWINPUT*) buf;
                // use input->data.mouse or input->data.hid as needed...
            }

            delete[] buf;
        }
    }

    TForm::WndProc(Message);
}

If you are using FireMonkey, there is no WndProc() method for handling window messages (FireMonkey does not dispatch window messages to user code at all). However, you can subclass the window that FireMonkey creates internally so you can still receive the WM_INPUT message. A static method is needed, but you do not have to rely on a global pointer, the Form object can be passed as a parameter of the subclassing:

class TMainForm : public TForm
{
private:
    static LRESULT CALLBACK SubclassProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam, UINT_PTR  uIdSubclass, DWORD_PTR dwRefData);
protected:
    virtual void __fastcall CreateHandle();
};

void __fastcall TMainForm::CreateHandle()
{
    TForm::CreateHandle();

    HWND hWnd = Platform::Win::WindowHandleToPlatform(this->Handle)->Wnd;

    SetWindowSubclass(hWnd, &SubclassProc, 1, (DWORD_PTR)this);

    RAWINPUTDEVICE Device = {0};
    Device.usUsagePage =  0x01;
    Device.usUsage = 0x02;
    Device.dwFlags = RIDEV_INPUTSINK;
    Device.hwndTarget = hWnd;

    RegisterRawInputDevices(&Device, 1, sizeof(RAWINPUTDEVICE));
}

LRESULT CALLBACK TMainForm::SubclassProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam, UINT_PTR  uIdSubclass, DWORD_PTR dwRefData)
{
    TMainForm *pThis = (TMainForm*) dwRefData;

    switch (uMsg)
    {
        case WM_INPUT:
        {
            // ...
            break;
        }

        case WM_NCDESTROY:
        {
            RemoveWindowSubclass(hWnd, &SubclassProc, uIdSubclass);
            break;
        }
    }

    return DefSubclassProc(hWnd, uMsg, wParam, lParam);
}

Answer 2

I came across the same issue and I found that the best method, for my particular case, was to create a static array of pointers to my class. Then inside the static hook method, I just iterate through my class pointers and call their hook functions.

kb_hook.h

typedef KBDLLHOOKSTRUCT khookstruct;
typedef LRESULT lresult;
typedef void (*h_func)(uint64_t key_message, khookstruct* kbdhook);
typedef std::vector<kb_hook*> h_array;

class kb_hook
{
public:
    kb_hook();
    virtual ~kb_hook();

    h_func hook_func;

private:
    static h_array hook_array;

    static lresult static_hook(int code, uint64_t key_message, khookstruct* kbdhook);
};

kb_hook.cpp

kb_hook::kb_hook() : hook_func(NULL)
{
    this->hook_array.push_back(this);
}

lresult kb_hook::static_hook(int code, uint64_t key_message, khookstruct* kbdhook)
{
    if(code == HC_ACTION)
    {
        for(auto it=kb_hook::hook_array.begin();it!=kb_hook::hook_array.end();it++)
        {
            if((*it)->hook_func) std::thread((*it)->hook_func, key_message, kbdhook).detach();
        }
    }

    return CallNextHookEx(NULL, code, key_message, reinterpret_cast<LPARAM>(kbdhook));
}

I know it's an old question but I just wanted to throw in my two cents. I hope this is helpful to someone.