2

Is there a proof-of-concept Objective-C executable that enters some text into an application and then clicks the mouse, using Apple events and not AppleScript?

e.g. the AppleScript equivalent of 

tell application "System Events"
 tell process "Safari"
  keystroke "Hello World"
  click
 end tell 
end tell

It should work on Mac OS X 10.9, preferably be future oriented (backwards compatibility doesn't matter). The context is that I will be calling the Objective-C code from another language.

I'm saying this because I read that:

As of Mac OS X 10.7, the low-level Cocoa API (NSAppleEventDescriptor) still lacks essential functionality (e.g. the ability to send Apple events), while the high-level Cocoa API (Scripting Bridge) is too flawed and limited to be a viable foundation for an appscript-style wrapper.

and:

NSAppleScript can safely be used only on the main thread

so, my goals are:

  1. any application (by name or if current)
  2. any keyboard input or mouse
  3. from C or Objective-C
  4. within a few hundred milliseconds

thanks!

sam boosalis
  • 1,997
  • 4
  • 20
  • 32

3 Answers3

3

Rather than using AppleEvents, the CGEvent API in CoreGraphics framework <https://developer.apple.com/library/mac/documentation/Carbon/Reference/QuartzEventServicesRef/Reference/reference.html> lets you post low-level mouse and keyboard events to the window server.

#include <CoreGraphics/CoreGraphics.h>

NSArray *launchedApplications = [[NSWorkspace sharedWorkspace] launchedApplications]; // depreciated but I couldn't find a modern way to get the Carbon PSN
NSPredicate *filter = [NSPredicate predicateWithFormat:@"NSApplicationName = \"TextEdit\""];
NSDictionary *appInfo = [[launchedApplications filteredArrayUsingPredicate:filter] firstObject];
ProcessSerialNumber psn;
psn.highLongOfPSN = [[appInfo objectForKey:@"NSApplicationProcessSerialNumberHigh"] unsignedIntValue];
psn.lowLongOfPSN = [[appInfo objectForKey:@"NSApplicationProcessSerialNumberLow"] unsignedIntValue];

CGEventRef event1 = CGEventCreateKeyboardEvent(NULL, (CGKeyCode)6, true); // 'z' key down
CGEventRef event2 = CGEventCreateKeyboardEvent(NULL, (CGKeyCode)6, false); // 'z' key up

CGEventPostToPSN(&psn, event1);
CGEventPostToPSN(&psn, event2);

You might also consider writing a Service <https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/SysServices/introduction.html>, which lets you provide functionality to other applications through the Service menu in the application menu. Note that you can even assign keyboard shortcuts to Service menu items. Services work via the system pasteboard; this approach may be easier than dealing with raw window server events if you simply need to be able to paste some canned or generated data into another application.

jrc
  • 20,354
  • 10
  • 69
  • 64
  • Thanks, I got this running. This seems like the right API. How do you send a mouse click? – sam boosalis Sep 23 '14 at 09:47
  • 1
    Use `CGEventCreateMouseEvent()`. See `CGEvent.h` (File menu > Open Quickly… is your friend) or the documentation linked above. – jrc Sep 23 '14 at 20:25
1

The best way for achieving your result is using Automator,

See https://developer.apple.com/library/mac/documentation/AppleApplications/Conceptual/AutomatorConcepts/AutomatorConcepts.pdf

If you want to achieve this through ObjectiveC, you need to understand "Distributed Objects Architecture". By pairing NSPort and NSInvocation, you can do amazing things, like cross-process and cross-machine method calling.

Here is a guide for that
https://developer.apple.com/librarY/prerelease/mac/documentation/Cocoa/Conceptual/DistrObjects/Concepts/architecture.html

l0gg3r
  • 8,864
  • 3
  • 26
  • 46
0

I'm not sure if this is what you're looking for, but you may be interested in setting up an NSInvocation object:

- (void)invokeWithTarget:(id)anObject

If you're looking to run some code and 'simulate' a UX environment, it may be valuable to save an invocation and run it.

(Automator?)

Brooks Hanes
  • 425
  • 2
  • 12
  • 1
    Still new to Objective-C, but how would you use this to send some text message to some application object? – sam boosalis Sep 18 '14 at 22:14
  • 1
    Answer here is going to be very helpful to see if invocation is right for your situation: http://stackoverflow.com/questions/313400/nsinvocation-for-dummies – Brooks Hanes Sep 19 '14 at 16:22
  • 1
    yeah I'd read that, hence my comment. Seems like a general message passing thing, rather than sending a specific Apple event message to a specific application object, in some other process. – sam boosalis Sep 19 '14 at 18:06
  • Have you looked into Automator? Thought I'd ask. – Brooks Hanes Sep 19 '14 at 18:11
  • 1
    yeah but I want to construct these actions programmatically, and access Objective-C directly. – sam boosalis Sep 19 '14 at 19:08