Using jna to consume key events

Question

In one of the applications I am writing i need to consume certain key events so other applications dont process them.

In my code i make a com.sun.jna.platform.win32.WinUser.LowLevelKeyboardProc like so:

import com.sun.jna.Native;
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.WinDef.HMODULE;
import com.sun.jna.platform.win32.WinDef.LRESULT;
import com.sun.jna.platform.win32.WinDef.WPARAM;
import com.sun.jna.platform.win32.WinUser.HHOOK;
import com.sun.jna.platform.win32.WinUser.KBDLLHOOKSTRUCT;
import com.sun.jna.platform.win32.WinUser.LowLevelKeyboardProc;
import com.sun.jna.platform.win32.User32;
import com.sun.jna.platform.win32.WinUser;

public class KeyHook implements Runnable{

private static volatile boolean quit = false;
private static HHOOK hhk;
private static LowLevelKeyboardProc keyboardHook;

private Main main;
User32 lib;
HMODULE hMod;
public boolean isHooked = false;

public KeyHook(final Main main) {
    this.main = main;
    lib = User32.INSTANCE;
    hMod = Kernel32.INSTANCE.GetModuleHandle(null);
    Native.setProtected(true);
}

@Override
public void run() {
    keyboardHook = new LowLevelKeyboardProc() {
        public LRESULT callback(int nCode, WPARAM wParam, KBDLLHOOKSTRUCT info) {
            if (nCode >= 0 && main.getPane().getTabCount() > 0) {
                switch (wParam.intValue()) {
                    case WinUser.WM_KEYUP:
                        if(info.vkCode == main.getListenMouse()){
                            main.listen();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getStopListenMouse()){
                            main.stopListening();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getStart()){
                            main.start();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getPause()){
                            main.pause();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getStop()){
                            main.stopRunning();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == 0x7B){
                            main.nextTab();
                            return new LRESULT(1);
                        }
                        break;
                    case WinUser.WM_KEYDOWN:
                       break;
                    case WinUser.WM_SYSKEYUP:
                        break;
                    case WinUser.WM_SYSKEYDOWN:
                        quit = true;
                        break;
                }
            }
            return lib.CallNextHookEx(hhk, nCode, wParam, info.getPointer());
            //return new LRESULT(1);
        }
    };
    hhk = lib.SetWindowsHookEx(WinUser.WH_KEYBOARD_LL, keyboardHook, hMod, 0);
}
}

When I return a new LRESULT(1) at the end of my proc (commented out code at the end), all keyboard events are consumed. However, when i replace it with

return lib.CallNextHookEx(hhk, nCode, wParam, info.getPointer());

as it should be, and only try to consume the main keyboard events i want to consume, it doesn't consume any of the keyboard events. Does anyone have any idea of why it won't let me consume the events I want or have any idea how to fix it so it will?

You need to call the next hook only when you *don't* want to consume an event, and return LRESULT(1) when you *do*. Perhaps the code that you intent to flag consumable events isn't actually being called? — technomage, Jan 16 '12 at 12:38
See http://stackoverflow.com/questions/8540891/using-jna-to-keyhook-and-consume — technomage, Jan 16 '12 at 12:45
@technomage: I do return LRESULT(1) when i want to consume it for individualy events. What im sayin is that it won't consume it when i do it for the individual events. And that question you made a link to was my question from a while ago, it solves it when i replace the nextHook with LRESULT(1), but when doing that it consumes all key events. — Michael Rentmeister, Jan 17 '12 at 07:34
Have you examined the code flow to determine whether the key input you want to consume (the cases where you return LRESULT(1)) is even appearing? — technomage, Jan 18 '12 at 02:17
yes, the key input is appearing and all code that is supposed to run as a result of the event are happening, but for some reason the events aren't being consumed afterwards — Michael Rentmeister, Jan 18 '12 at 08:37
Try installing a simplified hook which eats a single, well-defined keycode and passes on everything else. If that fails to work, then you can be pretty sure the problem is independent of your code. — technomage, Jan 18 '12 at 20:56
Okay i tested it and the problem is independent from my code =/ Any ideas from there? — Michael Rentmeister, Jan 19 '12 at 08:15
So when you get the well-defined keycode and return LRESULT(1), the original application still gets the WM_KEYUP event? Note that the original application may well be listening for WM_KEYDOWN or WM_CHAR, so you'd need to prevent those events from propagating as well. — technomage, Jan 20 '12 at 16:47
Ah thanks! You're a genius! I made it so i check the key outside of the "switch (wParam.intValue())" and set a flag if it's one of my hot keys, if it is then i return new LRESULT(1) no matter what the wParam is. Thanks a ton! — Michael Rentmeister, Jan 23 '12 at 05:47

score 2 · Accepted Answer · answered Jan 24 '12 at 15:28

In order to ensure that a key is "consumed", you need to ensure that you avoid calling the next hook (i.e. return LRESULT(1)) on all event variants of a given key, i.e. WM_KEYUP, WM_KEYDOWN, and possibly WM_CHAR.

Some applications may look for key up events, others for key down, and others simply for the produced character output, so you must consume all events related to a given keystroke to make said keystroke properly "disappear".

Using jna to consume key events

1 Answers1

Linked