How to detect global mouse button events

Question

I'm wondering how can I write a code to monitor the mouse buttons globally. This would be for OS X, and I'd like to try writing it in Qt/C++.

To begin with I don't know how to capture those global events. The monitor application would not display a GUI, it'd simply be a process that runs in the background and detects mouse buttons being clicked.

In the second part of the program I would like to launch hot-keys depending of the mouse key pressed.

My final idea is make a free program like steerMouse, just to figure out how it could be done.

I'm asking for a guidance of where to start - how can I detect the mouse button events globally?

thanks Kuba for correct my original question, sorry for my english, still learning. — Angoll, Oct 08 '13 at 12:06

score 8 · Accepted Answer · edited May 23 '17 at 12:31

It's not possible using only Qt. There's another question that details the issues. It boils down to:

Installing an event filter on QApplication will let you receive mouse events while the cursor is over any application window, but not outside of it. That's not helpful in your case.
If a widget grabs the mouse using grabMouse(), it will receive all mouse events globally, but interaction with other applications becomes impossible.

So, you'll need to resort to using platform-specific APIs to do this - that means Cocoa and writing in Objective C/C++. There's a question with excellent answers that provides almost everything we need but Qt integration.

The missing part, shown below, is integrating the stand-alone code with Qt. This code shows an empty widget just to demonstrate that we correctly handle mouse events for both our application, and outside of it.

This is a complete, working example, using Cocoa. It needs to go into a .mm file; don't forget to add it to OBJECTIVE_SOURCES in your qmake project file (not to SOURCES!).

Unfortunately, there's isn't a single function/method that would translate from NSEvent to QMouseEvent. The best one can do is copy&paste some code from qnsview.mm. This is unfortunate but results from the design of Qt platform abstraction: the platform code ends up calling QWindowSystemInterface::handleMouseEvent(....) to post the event to the application.

#include <QApplication>
#include <QAbstractNativeEventFilter>
#include <QTextStream>
#include <QWidget>
#include <cstdio>
#import <AppKit/AppKit.h>

QTextStream out(stdout);

class MyEventFilter : public QAbstractNativeEventFilter {
public:
    bool nativeEventFilter(const QByteArray &eventType, void *message, long *result) {
        Q_UNUSED(eventType) Q_UNUSED(result)
        NSEvent * event = (NSEvent*)message;
        switch ([event type]) {
        case NSLeftMouseDown:
            out << "Lv"; break;
        case NSLeftMouseUp:
            out << "L^"; break;
        case NSRightMouseDown:
            out << "Rv"; break;
        case NSRightMouseUp:
            out << "R^"; break;
        case NSOtherMouseDown:
            out << [event buttonNumber] << "v"; break;
        case NSOtherMouseUp:
            out << [event buttonNumber] << "^"; break;
        default:
            return false;
        }
        out << endl;
        return false;
    }
};

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);
    QSharedPointer<QAbstractNativeEventFilter> filter(new MyEventFilter);
    const int mask =
            NSLeftMouseDownMask | NSLeftMouseUpMask |
            NSRightMouseDownMask | NSRightMouseUpMask |
            NSOtherMouseDownMask | NSOtherMouseUpMask;
    // The global monitoring handler is *not* called for events sent to our application
    id monitorId = [NSEvent addGlobalMonitorForEventsMatchingMask:mask handler:^(NSEvent* event) {
        filter->nativeEventFilter("NSEvent", event, 0);
    }];
    // We also need to handle events coming to our application
    a.installNativeEventFilter(filter.data());
    QWidget w;
    w.show();
    int rc = a.exec();
    [NSEvent removeMonitor:monitorId];
    return rc;
}

Thanks Kuba, for the answer. Now I understand more how the events works in MacOs and I have an early version of what I want to do :) — Angoll, Oct 08 '13 at 12:03

phyatt · Answer 2 · 2013-10-07T18:06:50.740

Sounds like you want to hook global mouse events on OSX.

I've done it in Windows, with great success. I know what to look for.

Here is the best stuff I could find on it after a quick search:

https://code.google.com/p/jnativehook/

https://code.google.com/p/jnativehook/source/browse/branches/1.1/src/native/osx/NativeThread.c

Basically the JNativeHook does the following:

It creates a c thread with the correct callback to the system functions that handle the mouse. As the mouse (and keyboard) are handled by the system, the callback, gets the information. The information then gets forwarded on into the java side of the code through a call back.

You need to create a thread, hook it to the system properly, and then get the information out to where you want to log or display it. Over 90% of that work is accomplished in the NativeThread.c link above. Here are some key parts to it.

Lines 305 to 552 have the following:

switch (type) {
//...
case kCGEventLeftMouseDown:
        button = kVK_LBUTTON;
        SetModifierMask(kCGEventFlagMaskButtonLeft);
        goto BUTTONDOWN;

case kCGEventRightMouseDown:
        button = kVK_RBUTTON;
        SetModifierMask(kCGEventFlagMaskButtonRight);
        goto BUTTONDOWN;

case kCGEventOtherMouseDown:
        button = CGEventGetIntegerValueField(event, kCGMouseEventButtonNumber);

        if (button == kVK_MBUTTON) {
                SetModifierMask(kCGEventFlagMaskButtonCenter);
        }
        else if (button == kVK_XBUTTON1) {
                SetModifierMask(kCGEventFlagMaskXButton1);
        }
        else if (button == kVK_XBUTTON2) {
                SetModifierMask(kCGEventFlagMaskXButton2);
        }
BUTTONDOWN:
        #ifdef DEBUG
        fprintf(stdout, "LowLevelProc(): Button Pressed (%i)\n", (unsigned int) button);
        #endif

        // Track the number of clicks.
        #ifdef DEBUG
        fprintf(stdout, "LowLevelProc(): Click Time (%lli)\n", (CGEventGetTimestamp(event) - click_time)  / 1000000);
        #endif

        if ((long) (CGEventGetTimestamp(event) - click_time) / 1000000 <= GetMultiClickTime()) {
                click_count++;
        }
        else {
                click_count = 1;
        }
        click_time = CGEventGetTimestamp(event);

        event_point = CGEventGetLocation(event);
        jbutton = NativeToJButton(button);
        jmodifiers = NativeToJEventMask(GetModifiers());

        // Fire mouse pressed event.
        objMouseEvent = (*env)->NewObject(
                                                                env,
                                                                clsMouseEvent,
                                                                idMouseButtonEvent,
                                                                org_jnativehook_mouse_NativeMouseEvent_NATIVE_MOUSE_PRESSED,
                                                                (jlong) event_time,
                                                                jmodifiers,
                                                                (jint) event_point.x,
                                                                (jint) event_point.y,
                                                                (jint) click_count,
                                                                jbutton);
        (*env)->CallVoidMethod(env, objGlobalScreen, idDispatchEvent, objMouseEvent);
        (*env)->DeleteLocalRef(env, objMouseEvent);
        break;

case kCGEventLeftMouseUp:
        button = kVK_LBUTTON;
        UnsetModifierMask(kCGEventFlagMaskButtonLeft);
        goto BUTTONUP;

case kCGEventRightMouseUp:
        button = kVK_RBUTTON;
        UnsetModifierMask(kCGEventFlagMaskButtonRight);
        goto BUTTONUP;

case kCGEventOtherMouseUp:
        button = CGEventGetIntegerValueField(event, kCGMouseEventButtonNumber);

        if (button == kVK_MBUTTON) {
                UnsetModifierMask(kCGEventFlagMaskButtonCenter);
        }
        else if (button == kVK_XBUTTON1) {
                UnsetModifierMask(kCGEventFlagMaskXButton1);
        }
        else if (button == kVK_XBUTTON2) {
                UnsetModifierMask(kCGEventFlagMaskXButton2);
        }

BUTTONUP:
        #ifdef DEBUG
        fprintf(stdout, "LowLevelProc(): Button Released (%i)\n", (unsigned int) button);
        #endif

        event_point = CGEventGetLocation(event);
        jbutton = NativeToJButton(button);
        jmodifiers = NativeToJEventMask(GetModifiers());

        // Fire mouse released event.
        objMouseEvent = (*env)->NewObject(
                                                                env,
                                                                clsMouseEvent,
                                                                idMouseButtonEvent,
                                                                org_jnativehook_mouse_NativeMouseEvent_NATIVE_MOUSE_RELEASED,
                                                                (jlong) event_time,
                                                                jmodifiers,
                                                                (jint) event_point.x,
                                                                (jint) event_point.y,
                                                                (jint) click_count,
                                                                jbutton);
        (*env)->CallVoidMethod(env, objGlobalScreen, idDispatchEvent, objMouseEvent);
        (*env)->DeleteLocalRef(env, objMouseEvent);

        if (mouse_dragged != true) {
                // Fire mouse clicked event.
                objMouseEvent = (*env)->NewObject(
                                                                        env,
                                                                        clsMouseEvent,
                                                                        idMouseButtonEvent,
                                                                        org_jnativehook_mouse_NativeMouseEvent_NATIVE_MOUSE_CLICKED,
                                                                        (jlong) event_time,
                                                                        jmodifiers,
                                                                        (jint) event_point.x,
                                                                        (jint) event_point.y,
                                                                        (jint) click_count,
                                                                        jbutton);
                (*env)->CallVoidMethod(env, objGlobalScreen, idDispatchEvent, objMouseEvent);
                (*env)->DeleteLocalRef(env, objMouseEvent);
        }
        break;


case kCGEventLeftMouseDragged:
case kCGEventRightMouseDragged:
case kCGEventOtherMouseDragged:
        event_point = CGEventGetLocation(event);

        #ifdef DEBUG
        fprintf(stdout, "LowLevelProc(): Motion Notified (%f, %f)\n", event_point.x, event_point.y);
        #endif

        // Reset the click count.
        if (click_count != 0 && (long) (CGEventGetTimestamp(event) - click_time) / 1000000 > GetMultiClickTime()) {
                click_count = 0;
        }
        jmodifiers = NativeToJEventMask(GetModifiers());

        // Set the mouse dragged flag.
        mouse_dragged = true;

        // Fire mouse dragged event.
        objMouseEvent = (*env)->NewObject(
                                                                env,
                                                                clsMouseEvent,
                                                                idMouseMotionEvent,
                                                                org_jnativehook_mouse_NativeMouseEvent_NATIVE_MOUSE_DRAGGED,
                                                                (jlong) event_time,
                                                                jmodifiers,
                                                                (jint) event_point.x,
                                                                (jint) event_point.y,
                                                                (jint) click_count);
        (*env)->CallVoidMethod(env, objGlobalScreen, idDispatchEvent, objMouseEvent);
        (*env)->DeleteLocalRef(env, objMouseEvent);
        break;

case kCGEventMouseMoved:
        event_point = CGEventGetLocation(event);
        #ifdef DEBUG
        fprintf(stdout, "LowLevelProc(): Motion Notified (%f, %f)\n", event_point.x, event_point.y);
        #endif

        // Reset the click count.
        if (click_count != 0 && (long) (CGEventGetTimestamp(event) - click_time) / 1000000 > GetMultiClickTime()) {
                click_count = 0;
        }
        jmodifiers = NativeToJEventMask(GetModifiers());

        // Set the mouse dragged flag.
        mouse_dragged = false;

        // Fire mouse moved event.
        objMouseEvent = (*env)->NewObject(
                                                                env,
                                                                clsMouseEvent,
                                                                idMouseMotionEvent,
                                                                org_jnativehook_mouse_NativeMouseEvent_NATIVE_MOUSE_MOVED,
                                                                (jlong) event_time,
                                                                jmodifiers,
                                                                (jint) event_point.x,
                                                                (jint) event_point.y,
                                                                (jint) click_count);
        (*env)->CallVoidMethod(env, objGlobalScreen, idDispatchEvent, objMouseEvent);
        (*env)->DeleteLocalRef(env, objMouseEvent);
        break;

case kCGEventScrollWheel:
        event_point = CGEventGetLocation(event);

        // TODO Figure out of kCGScrollWheelEventDeltaAxis2 causes mouse events with zero rotation.
        if (CGEventGetIntegerValueField(event, kCGScrollWheelEventIsContinuous) == 0) {
                jscrollType = (jint)  org_jnativehook_mouse_NativeMouseWheelEvent_WHEEL_UNIT_SCROLL;
        }
        else {
                jscrollType = (jint)  org_jnativehook_mouse_NativeMouseWheelEvent_WHEEL_BLOCK_SCROLL;
        }

        // Scrolling data uses a fixed-point 16.16 signed integer format (Ex: 1.0 = 0x00010000).
        jwheelRotation = (jint) CGEventGetIntegerValueField(event, kCGScrollWheelEventDeltaAxis1) * -1;

        /* TODO Figure out the scroll wheel amounts are correct.  I
        * suspect that Apples Java implementation maybe reporting a
        * static "1" inaccurately.
        */
        jscrollAmount = (jint) CGEventGetIntegerValueField(event, kCGScrollWheelEventPointDeltaAxis1) * -1;

        #ifdef DEBUG
        fprintf(stdout, "LowLevelProc(): Mouse Wheel Moved (%i, %i, %i)\n", (int) jscrollType, (int) jscrollAmount, (int) jwheelRotation);
        #endif

        // Track the number of clicks.
        if ((long) (CGEventGetTimestamp(event) - click_time) / 1000000 <= GetMultiClickTime()) {
                click_count++;
        }
        else {
                click_count = 1;
        }
        click_time = CGEventGetTimestamp(event);

        jmodifiers = NativeToJEventMask(GetModifiers());

        // Fire mouse wheel event.
        objMouseWheelEvent = (*env)->NewObject(
                                                                        env,
                                                                        clsMouseWheelEvent,
                                                                        idMouseWheelEvent,
                                                                        org_jnativehook_mouse_NativeMouseEvent_NATIVE_MOUSE_WHEEL,
                                                                        (jlong) event_time,
                                                                        jmodifiers,
                                                                        (jint) event_point.x,
                                                                        (jint) event_point.y,
                                                                        (jint) click_count,
                                                                        jscrollType,
                                                                        jscrollAmount,
                                                                        jwheelRotation);
        (*env)->CallVoidMethod(env, objGlobalScreen, idDispatchEvent, objMouseWheelEvent);
        (*env)->DeleteLocalRef(env, objMouseWheelEvent);
        break;

#ifdef DEBUG
default:
        fprintf(stderr, "LowLevelProc(): Unhandled Event Type: 0x%X\n", type);
        break;
#endif
}

That should get you started.

Hope that helps.

-1 for being only tangentially related. Your code snippet is mostly unhelpful. — Kuba hasn't forgotten Monica, Oct 07 '13 at 19:07
Based on the two caveats at the top of your answer, I'd call your answer unhelpful, too. Personally, I'd let the asker, or other third parties decide which is more useful. Let's see what the asker thinks when he gets into it. :) BTW, I've done a lot of what you mentioned in your answer in the past, and I ended needing OS system wide hooks. — phyatt, Oct 07 '13 at 19:22
Thanks for the answer, I find both pretty interesting, this one because I never been heard about the Hooks and about the library. I will keep in mind if I want to extend the program to other OS. — Angoll, Oct 08 '13 at 12:00

How to detect global mouse button events

2 Answers2

Linked