1

You can list the currently open windows using CGWindowListCreate, e.g:

CFArrayRef windowListArray = CGWindowListCreate(kCGWindowListOptionOnScreenOnly|kCGWindowListExcludeDesktopElements, kCGNullWindowID);
NSArray *windows = CFBridgingRelease(CGWindowListCreateDescriptionFromArray(windowListArray));

// stuff here with windows array

CFRelease(windowListArray);

With this you can get a specific window, e.g. a window from Chrome that's somewhere in the background of the current workspace, but not minimized. I also found that you can simulate mouse-clicks anywhere on-screen using CGEventCreateMouseEvent:

CGEventRef click_down = CGEventCreateMouseEvent(NULL, kCGEventLeftMouseDown, point, kCGMouseButtonLeft);
CGEventPost(kCGHIDEventTap, click_down);

Instead of sending this event to the front-most window under this point on the screen, can I send this to a window in background? Or is the only way to temporarily switch to that window (bring it to front), click, and switch back to the previous frontmost window?

This post suggests the latter is possible: Cocoa switch focus to application and then switch it back

Although I'm very interested in seeing whether this can be avoided, whether we can send mouse-clicks directly to a specific window in background without bringing it into view. I can consider any Objective-C, C, or C++ options for this.

Community
  • 1
  • 1
Tiago
  • 1,984
  • 1
  • 21
  • 43
  • Do you mean you want to simulate a click on a part of the window which is obscured behind other windows? If it's not obscured, then there's no problem. Posting the mouse-down at a position over a revealed part of the window will cause it to be routed to that window. Otherwise, you can try `CGEventPostToPSN()` to post it directly to the owning app process, although it will still go to the window within that app that's front-most under the event position. – Ken Thomases Apr 09 '17 at 23:41
  • The idea is here is that it's obscured. That way I can automate a task (some UI clicks in a window obscured by my current window), without it interrupting my main task. I'm looking into CGEventPostToPSN. It looks like GetProcessForPID is deprecated? Is there a different way for getting the ProcessSerialNumber for a process? – Tiago Apr 09 '17 at 23:48
  • I don't think there's a modern replacement. I'd just use `GetProcessForPID()` and, if you find it works and you're relying on it, file a bug with Apple asking for an appropriate replacement for `CGEventPostToPSN()`. – Ken Thomases Apr 10 '17 at 00:44
  • CGEventPostToPSN is working very well, though I feel stuck in that GetProcessForPID is deprecated. NSWorkspace launchedApplications also gives access to the serialnumberLow and High values needed for the PSN struct, but that method is deprecated too, in favour of runningApplications, which doesn't have that information – Tiago Apr 10 '17 at 09:28
  • Does the app process background clicks? I don't know about Chrome but Safari and FireFox don't. – Willeke Apr 10 '17 at 09:38
  • @Willeke I've got this working with Chrome in the background, using a browser based whiteboard to test with. The window is in background, with Xcode and my app in front, and mouse down and up events are drawing on the window in background. Haven't tried safari and firefox though. Either way looking for alternatives that aren't deprecated, cautious of building on deprecated APIs. – Tiago Apr 10 '17 at 09:44
  • @KenThomases thanks for the comments, found that there's a CGEventPostToPid function already. No need for GetProcessForPID. CGEventPostToPSN should really be marked as deprecated, since all other PSN functions seem to be. – Tiago Apr 10 '17 at 10:08
  • @Tiago, interesting. I wasn't aware of `CGEventPostToPid()`. Nice. – Ken Thomases Apr 10 '17 at 14:02

1 Answers1

2

Finally got this working after finding an alternative to the deprecated GetPSNForPID function:

NSEvent *customEvent = [NSEvent mouseEventWithType: NSEventTypeLeftMouseDown
                                          location: point
                                     modifierFlags: NSEventModifierFlagCommand
                                         timestamp:[NSDate timeIntervalSinceReferenceDate]
                                      windowNumber:[self.windowID intValue]
                                           context: nil
                                       eventNumber: 0
                                        clickCount: 1
                                          pressure: 0];

CGEvent = [customEvent CGEvent];
CGEventPostToPid(PID, CGEvent);

Didn't realise there was a CGEventPostToPid method instead of CGEventPostToPSN. No need for the ProcessSerialNumber struct.

Tiago
  • 1,984
  • 1
  • 21
  • 43
  • Were you able to ever actually send mouse events to non-focused apps? – user1768741 Jul 23 '17 at 01:35
  • Unreliably, but I don't know if my problem was that the app I was targeting was a flash-based app in a browser window. I was testing with a drawing app built in javascript, and that was drawing well while in background, but once I tried to test it with the flash-based app I just couldn't get it to trigger background-clicks. @user1768741 – Tiago Jul 24 '17 at 09:54