0

I work on a product that uses accessories to send commands to the system. These accessories are basically USB keyboards and each of them has a different set of keys, on which we must act differently, depending on which accessory sent the keys. Plus, we need to make sure that the rest of the OS does not receive the key events from the accessories when our system is running.

After some research, I found this article that helped me have a working solution for one accessory at a time, using and combining Raw Input and Windows Hooks. While the article uses C++ for the full solution, I use C++ for the Windows Hooks portion (global hooks require the code to be in a native language) and C# for the Raw Input processing + decision making. So my managed application will load the unmanaged code and install the WH_KEYBOARD hook, setting up a message code (based off of WM_APP). The hook callback will send a message to the calling app, which in turn, based on the raw input for the same key, will do two things: communicate the main app according to the command/key pressed; and return to the hook callback if the input should be passed through (no command/key pressed) - by calling CallNextHook, or blocked - by returning 1.

As I've mentioned, it works as expected for one accesory. But once I run a second process, with the only difference being the message code, the first process stops receiving the messages from SendMessage and the second process starts receiving the messages as intended. From the Microsoft oficial documentation on hooks, any app that installs a type of hook will have its callback function placed on that particular hook type hook chain and, therefore, be processed as expected.

The native code (NativeHook.dll) goes like this:

LRESULT CALLBACK KeyboardProc(int nCode, WPARAM wParam, LPARAM lParam)
{
    fprintf(stdout, "proc\n");
    if (nCode < 0)
    {
        return CallNextHookEx(hookHandle, nCode, wParam, lParam);
    }
    USHORT keyPressed = lParam & 0x80000000 ? 0 : 1;
    if (SendMessage(windowHandle, MSGID, wParam, keyPressed))
    {
        fprintf(stdout, "block\n");
        return TRUE;
    }
    fprintf(stdout, "next\n");

    return CallNextHookEx(hookHandle, nCode, wParam, lParam);

}


BOOL HookUp(HWND handle, UINT msgid, DWORD threadId)
{
    if (windowHandle != NULL)
    {
        // already hooked
        return 2;
    }

    hookHandle = SetWindowsHookEx(WH_KEYBOARD, (HOOKPROC)KeyboardProc, dllInstance, threadId);
    if (hookHandle == NULL)
    {
        DWORD errorCode = GetLastError();
        fprintf(stdout, "hooking error code: %d\n", errorCode);
        return errorCode;
    }

    MSGID = msgid;
    windowHandle = handle;
    fprintf(stdout, "hook handle: %p; window handle: %p; MSGID: %d\n", hookHandle, windowHandle, MSGID);
    
    return TRUE;
}

BOOL WINAPI DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
{
    switch (fdwReason)
    {
    case DLL_PROCESS_ATTACH:
        dllInstance = hinstDLL;
        AllocConsole(); // Enable the console
        freopen_s((FILE**)stdout, "CONOUT$", "w", stdout);
        freopen_s((FILE**)stdin, "CONIN$", "r", stdin);
        fprintf(stdout, "%p\n", hinstDLL);
        break;
    case DLL_PROCESS_DETACH:
        UnhookWindowsHookEx(hookHandle);
        fprintf(stdout, "%p detached (Window: %p)\n", dllInstance, windowHandle);
        break;
    default:
        break;
    }

    return TRUE;

}

The relevant portions of the managed code goes like this:

[DllImport("user32.dll", SetLastError = true)]
public static extern int GetWindowThreadProcessId(IntPtr hWnd, IntPtr processId);

[DllImport("NativeHook.dll", EntryPoint = "HookUp", SetLastError = true, CallingConvention = CallingConvention.Cdecl)]
public static extern int HookUp(IntPtr appHandle, uint msgid, uint threadId);

// zero for the window handle and the process ID will install the hook globally 
int threadId = NativeResources.GetWindowThreadProcessId(IntPtr.Zero, IntPtr.Zero);

var hooked = NativeResources.HookUp(this.Handle, WM_APP + (uint)MessageId, (uint)threadId);

// ...

protected override void WndProc(ref Message m)
{
    if (m.Msg == WM_INPUT)
    {
        // raw input will allow the identification of which keyboard generated the input
        base.WndProc(ref m);
    }
    // MessageId is a property initialized based on the device to be monitored
    else if (m.Msg == WM_APP + MessageId)
    {
        IntPtr intercept = new IntPtr(1);
        // the value for _block will depend on a logic that will check if there's a raw input of the same key AND if that key was pressed from the desired keybaord
        if (_block)
        {
            // setting an IntPtr with 1 means block to the hook callback on the native code
            m.Result = intercept;
        }
        else
        {
            // calling base.WndProc means that the hook callback should call CallNextHook
            base.WndProc(ref m);
        }
    }
    else
    {
        base.WndProc(ref m);
    }
}

Even if I leave my managed code without any decision making process - only calling base.WndProc from both types of message, when the second process starts, the first process only receives WM_INPUT messages, no longer getting the WM_APP + MessageId messages.

With that behavior I'm not able to monitor and act on two or more accessories at a time, leaving my solution incomplete and, therefore, not suitable for our needs.

My question is: are there any particularities that I should apply when installing my hook in order for it to run as expected in as many processes as I need to be able to monitor various keyboards/accessories?

allw
  • 1
  • 1
  • 2
    If you properly handle WM_INPUT, calling GetRawInputData(), then you hijack the keyboard event early and won't get the WH_KEYBOARD hook callback. It is not obvious to me why it requires two processes to have that malfunction. Consider calling SendMessage() if the raw input is a match. – Hans Passant Dec 02 '22 at 22:31
  • 2
    How is `windowHandle` defined? If it's in shared data then your DLL will be sharing the one value in every process it's loaded into (so starting a new process will overwrite the value and the window handle from the original process will be lost). I'm assuming you're using `WM_KEYBOARD` as a global hook, in which case your DLL is loaded once into every process in the system. You may need to find the target window another way (e.g. by title/class using FindWindow) or use multiple hook procedures each with their own data to store the target window handle. – Jonathan Potter Dec 02 '22 at 22:31
  • @HansPassant I need two processes because some of our accessories use the same keys, but we treat them differently based on which device generated the input. Since I don't have a way to change the keys used by the accessories, I need to be able to act on each key from each device independently - hence the need for more than one process. From all my tests with only one process and one accessory, the fact that I use RawInput does not interfere with the hook callback being called. So the same behavior should apply to two processes. – allw Dec 05 '22 at 11:26
  • @JonathanPotter windowHandle is not in shared data. I've considered that this would be the cause of my issue, but for that I created a function that would print out the values of windowHandle and MSGID. I call this function from the managed code after 20s and after 40s for each process (the times are just for testing). The values of windowHandle and MSGID do not change and do not overlap from one process to the other. What changes is the fact that the hook callback no longer gets called on the first process once I start the second process. – allw Dec 05 '22 at 11:30

0 Answers0