Get window values under mouse

Question

I am playing with Cocoa/Objective-C and I would like to ask you if it is possible to get window information such as pid, window name from a window that is inactive. What I precisely mean is, in the case there are two non full screen (nor maximized) windows A and B of different tasks, lets say Chrome(A) and Firefox(B), with active window A and mouse cursor above window B, can I get such info without having to click into window B and bring it in foreground?

I have noticed that for example scrolling in an inactive window scrolls the context as if it was in foreground, so my guess is that this is doable.

Any tips would be really welcome.

Thank you

score 8 · Accepted Answer · edited Mar 15 '18 at 00:54

I will answer my own question: It possible with accessibility api & carbon a) register a wide event:

AXUIElementRef _systemWideElement = AXUIElementCreateSystemWide();

b)convert carbon to screen point

- (CGPoint)carbonScreenPointFromCocoaScreenPoint:(NSPoint)cocoaPoint {
    NSScreen *foundScreen = nil;
    CGPoint thePoint;

    for (NSScreen *screen in [NSScreen screens]) {
        if (NSPointInRect(cocoaPoint, [screen frame])) {
            foundScreen = screen;
        }
    }

    if (foundScreen) {
        CGFloat screenMaxY = NSMaxY([foundScreen frame]);

        thePoint = CGPointMake(cocoaPoint.x, screenMaxY - cocoaPoint.y - 1);
    } else {
        thePoint = CGPointMake(0.0, 0.0);
    }

    return thePoint;
}

c) get process under mouse

NSPoint cocoaPoint = [NSEvent mouseLocation];
if (!NSEqualPoints(cocoaPoint, _lastMousePoint)) {
    CGPoint pointAsCGPoint = [self carbonScreenPointFromCocoaScreenPoint:cocoaPoint];

    AXUIElementRef newElement = NULL;
    if (AXUIElementCopyElementAtPosition( _systemWideElement, pointAsCGPoint.x, pointAsCGPoint.y, &newElement ) == kAXErrorSuccess) {

        NSLog(@"%@",newElement);

    }
    _lastMousePoint = cocoaPoint;
}

Credits to https://developer.apple.com/library/mac/samplecode/UIElementInspector/Introduction/Intro.html

nslog gives something like <AXUIElement 0x6000000583c0> {pid=39429}

ps aux | grep 39429

:39429   0.2  5.5  5109480 916500   ??  U     1:57PM   3:34.67 /Applications/Xcode.app/Contents/MacOS/Xcode

Awesome and perfect. I often forget to refer to that sample app and forget how blunt the AX hammer is. — uchuugaka, Jul 07 '15 at 15:11
You can get the carbon pointer location directly using `CGEventRef e = CGEventCreate(NULL); CGPoint location = CGEventGetLocation(e); CFRelease(e)` — Noah Nuebling, Nov 11 '22 at 21:22
Also, this solution is quite slow and I think the Carbon screen point conversion isn't correct. — Noah Nuebling, May 06 '23 at 09:27

Noah Nuebling · Answer 2 · 2023-05-06T10:01:20.977

You can get the ID of the window under the mouse pointer using [NSWindow windowNumberAtPoint:belowWindowWithWindowNumber:]. Then you can use CGWindowListCopyWindowInfo() to get any info on the window. This is significantly faster than using the AXUI API mentioned in the other answer. I was using the AXUI API for my own project but it was so slow that it was sometimes causing hangs and other problems, so I don't recommend using it.

Here's my implementation for getting the app under the mouse pointer without using the slow AXUI API:

+ (NSRunningApplication * _Nullable)appUnderMousePointer:(NSPoint)pointerLocation {

    /// Get PID under mouse pointer
    
    pid_t pidUnderPointer = -1; /// I hope -1 is actually unused?
    
    CGWindowID windowNumber = (CGWindowID)[NSWindow windowNumberAtPoint:pointerLocation belowWindowWithWindowNumber:0];
    NSArray *windowInfo = (__bridge_transfer NSArray *)CGWindowListCopyWindowInfo(kCGWindowListOptionIncludingWindow, windowNumber);
    if (windowInfo.count > 0) {
        pidUnderPointer = [windowInfo[0][(__bridge NSString *)kCGWindowOwnerPID] intValue];
    }
    
    /// Get runningApplication

    NSRunningApplication *appUnderMousePointer = [NSRunningApplication runningApplicationWithProcessIdentifier:pidUnderPointer];
    
    return appUnderMousePointer;
}

You can get the pointerLocation easily using [NSEvent mouseLocation]. If you already have a CGEvent you can also get the pointerLocation using CGEventGetLocation(). Obtaining the location from the latest CGEvent instead of using [NSEvent mouseLocation] ensures that the location is up-to-date and should be more efficient. However, if you get the pointerLocation from a CGEvent, you need to flip the coordinates before plugging them into [NSWindow windowNumberAtPoint:belowWindowWithWindowNumber:]. That's because [NSWindow windowNumberAtPoint:belowWindowWithWindowNumber:] expects the pointerLocation to be given in Cocoa screen coordinates, while CGEventGetLocation() provides the pointer location in Quartz screen coordinates. Learn how to flip the coordinates here.

Get window values under mouse

2 Answers2

Linked