Swift macOS: How to paste into another application

Question

I'm writing an assistive application for Mac OS 10.10+, using swift. I need to be able to paste the content from the general NSPasteboard into the application which had previously been active.

Just to make it extra clear: I need to paste into another application.

It should work like this:

The user is using some random application
Because the user can't press cmd+v due to disability, they make a gesture which activates my app (this part is done and it's outside the scope of this question)
My app becomes active and now I need to simulate a Paste action in the app the user has been using beforehand. This is the bit I don't know how to do.
Finally the previously active app needs to become active again.

Please bare in mind the app is to be submitted to the AppStore.

Don't make your app active, you can't paste in an inactive app and the other app will blink. — Willeke, Oct 18 '16 at 16:41
[How to paste text from one app to another using Cocoa?](http://stackoverflow.com/questions/2680760/how-to-paste-text-from-one-app-to-another-using-cocoa) — Willeke, Oct 18 '16 at 22:11
@Willeke I'm afraid that the solution is not AppStore compatible. Also I do need to make my app active before the paste because the user needs to confirm the action before pasting. This is an accessibility requirement. — Neovibrant, Oct 21 '16 at 00:58
@Willeke There are other apps in the AppStore that exhibit similar functionality. So there must be a way to do it. — Neovibrant, Oct 21 '16 at 22:03
@Neovibrant: As a heads-up, just because there is another app in the Mac App Store with this functionality, does not mean your app will pass app review. (1): Some older Mac apps were submitted before sandboxing was a requirement. They are "grandfathered" in - as long as only bug fix updates are submitted, they can remain un-sandboxed (currently). (2): New apps **must** be sandboxed to be submitted to the Mac App Store. No exceptions. (3): It is possible that a developer "snuck" private API use or other functionality past app review. You can't count on being able to do the same. — breakingobstacles, Oct 22 '16 at 01:18
@Neovibrant: Furthermore, "you cannot sandbox an assistive app such as a screen reader, and you cannot sandbox an app that controls another app" [according to Apple's guide on "Determining Whether Your App Is Suitable for Sandboxing"](https://developer.apple.com/library/content/documentation/Security/Conceptual/AppSandboxDesignGuide/DesigningYourSandbox/DesigningYourSandbox.html#//apple_ref/doc/uid/TP40011183-CH4-SW6). They explicitly call out that "Use of accessibility APIs in assistive apps" and "Sending Apple events to arbitrary apps" are incompatible with the Mac App Sandbox. — breakingobstacles, Oct 22 '16 at 01:24
I'm pretty sure the NSUserAppleScriptTask I outline in my answer should work – it's executed out-of-process, outside the sandbox. I use a similar design in my sandboxed app Manuscripts (http://manuscriptsapp.com) to communicate with arbitrary other apps for the purpose of automating the importing of data, and I'm not listing those app bundle IDs that the app talks with in the entitlements, all working fine. — mz2, Oct 24 '16 at 09:58
@mz2 that sounds promising! Is you app using an entitlement to also write the scripts to the application's script directory? — Neovibrant, Oct 24 '16 at 12:04
No specific entitlement exists for that (beyond you needing the entitlement to read & write user selected files) - you need to request for write access to it by asking for a security scoped URL for that using the open dialog (all detailed in the objc.io article linked to from my answer). — mz2, Oct 24 '16 at 12:06
@mz2: Per the documentation on NSUserAppleScriptTask, "If the application is sandboxed, then the script must be in the applicationScriptsDirectory folder. A sandboxed application may read from, but not write to, this folder." The ability to request security scoped writing access through an open folder panel is due to an implementation detail that could always change to match the documentation (i.e. Apple could decide to block sandboxed apps from requesting access to this folder at some future date to enforce the documented restriction). Enabling core functionality with this is a risk, IMO. — breakingobstacles, Oct 24 '16 at 17:36
In any case, methods that work now aren't guaranteed to work in the future - especially if they go against the spirit of the sandboxing restrictions (Apple has closed workarounds in the past). It is important to reiterate that Apple's intent with Mac App Sandboxing (as explicitly documented) is that sending Apple events to arbitrary apps is incompatible with sandboxing. — breakingobstacles, Oct 24 '16 at 17:39
Again, all of this is described in the objc.io article – you need to request write access to that folder, and you will indeed get then write access to it. Without taking a position on how sensible I believe this scheme to be as a user experience, it is not against the spirit of the sandboxing restrictions, as noted in the objc.io article there are apps such as xScope in the Mac App Store which use this very trick and the user has full visibility to what the app is automating. The obvious downside to this is that the user technically has the capability of breaking the feature too. — mz2, Oct 24 '16 at 18:15
I of course agree that it's possible that this loop hole can be closed – at which point you may need to then ask the user to drag something into that folder to get the feature working again. I doubt this is anyway on the way out – it's been there since macOS 10.8, i.e. there's been plenty of chances for the sandbox to be hardened in this respect. — mz2, Oct 24 '16 at 18:17
Remember also that the script as executed from NSUserAppleScriptTask gets executed _outside_ the sandbox (that too is documented). I'm only guessing but would think that that's precisely there to allow for the scripts to access resources outside the sandbox. One can of course take an even more fatalist position and say that Apple Event delivery is generally diametrically opposed to the sandbox (it is). Should that prove right, almost certainly no method of automating user actions in arbitrary apps will survive (+ I'm pretty sure this is the only way to accomplish this behaviour inside sandbox) — mz2, Oct 24 '16 at 18:29
mz2: Indeed, my concern is that this method will not survive **for this purpose**. (But that largely depends on Apple and the precedence of the various clauses in the documentation.) In the most pessimistic reading, an app-supplied script is *not* a user-supplied script **(strike 1)**, NSUserAppleScriptTask "is not intended to execute scripts built into an application" **(strike 2)**, a sandboxed application may not write to the applicationScriptsDirectory folder **(strike 3)**. NSUserAppleScriptTask is for **user-supplied scripts**, not for application scripts - this loophole may be patched. — breakingobstacles, Oct 24 '16 at 18:43
We're going in circles: I understand it can disappear, at which point you can ask the user to follow (even more) contrived steps to put a script there. However, this is a pretty academic discussion in the sense that a) some means of relying on Apple Events to arbitrary apps is fundamentally ever the way to accomplish this inside the sandbox, b) my solution does work, c) it's worked for five major OS releases, d) others are doing it (xScope), e) should it stop working, you can ask the user place the right kind of script in the script folder. — mz2, Oct 25 '16 at 01:23
In a wider context than this question I do of course agree that I would consider twice whether an app which fundamentally needs to know about system state outside of its own sandbox (including the active applications) should be built for Mac App Store distribution; This particular problem can be solved and that is what we should be discussing here, but whether you want to go ahead with it depends on many factors beyond technology (is it worth the risk and hassle monetarily or otherwise – many devs' views differ on this). Goes way beyond this question though, and technically this is solvable. — mz2, Oct 25 '16 at 01:41
As an update, please see developer reports of App Review rejections for similar things here: https://forums.developer.apple.com/thread/67953 — breakingobstacles, Nov 24 '16 at 05:09

score 8 · Accepted Answer · edited May 23 '17 at 12:01

At its simplest you need to do two things:

Subscribe to notifications of the current application changing so you know what application you should send events to.
Simulate a Cmd+V keypress using System Events.

However, the bad news is, as you say you need to do this in an App Store submitted app, your app needs to be inside the sandbox and requires the temporary entitlement com.apple.security.temporary-exception.apple-events to send events to System Events directly quoting the Apple documentation:

requesting the necessary apple-events temporary exception entitlements for the Finder and System Events will likely result in rejection during the app review process, because granting access to these processes gives your app free rein over much of the operating system. For system-level tasks, use other methods, as discussed above.

There is also a way to do the step #2 using assistive technologies, but that too will not work inside the sandbox. Fundamentally, what you are trying to do (to activate an external arbitrary app, and to control it) is pretty much bang on what the sandbox is designed to prevent you from doing.

Fortunately, since macOS 10.8 there is now NSUserAppleScriptTask, which is executed outside of the sandbox.

There is a twist to NSUserAppleScriptTask: scripts executed with NSUserAppleScriptTask need to be placed inside the application's script directory, and your application cannot write to it, only read from it.

This objc.io article shows an example of how you can request for security scoped writing access to the scripting directory to be able to write your script to it; Annoyingly to yourself and your user you need to bring up an open dialog on the first time, therefore, and you need to store the security scoped bookmark so you don't need to repeat the exercise, but that's the best you'll be able to do inside the sandbox.

Your only other route beside jumping the NSUserAppleScriptTask hoops to my knowledge is to convince Apple that it's a really good idea to accept your app with the temporary entitlement that allows it to script System Events (I would not hold my breath).

Regarding NSUserAppleScriptTask and prompting the user to install a script from within your app, I just had my app rejected for doing that, citing section 2.4.5 of the App Store Review Guidelines so I guess we're out of luck :( — Wes, Dec 14 '18 at 07:08

score 8 · Answer 2 · edited Oct 20 '18 at 17:08

I found these commands to simulate cut/copy/paste using Foundation:

func pastematchstyle () {

    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x09, keyDown: true); // opt-shft-cmd-v down
    event1?.flags = [CGEventFlags.maskCommand, CGEventFlags.maskShift, CGEventFlags.maskAlternate]
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x09, keyDown: false); // opt-shf-cmd-v up
 //   event2?.flags = [CGEventFlags.maskCommand, CGEventFlags.maskShift, CGEventFlags.maskAlternate]
    event2?.post(tap: CGEventTapLocation.cghidEventTap);

}

func paste () {

    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x09, keyDown: true); // cmd-v down
    event1?.flags = CGEventFlags.maskCommand;
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x09, keyDown: false) // cmd-v up
//    event2?.flags = CGEventFlags.maskCommand
    event2?.post(tap: CGEventTapLocation.cghidEventTap)


}


func pasteresults () {

    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x09, keyDown: true); // shft-cmd-v down
    event1?.flags = [CGEventFlags.maskCommand, CGEventFlags.maskShift]
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x09, keyDown: false); // shf-cmd-v up
//    event2?.flags = [CGEventFlags.maskCommand, CGEventFlags.maskShift];
    event2?.post(tap: CGEventTapLocation.cghidEventTap);



}


func cut() {


    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x07, keyDown: true); // cmd-x down
    event1?.flags = CGEventFlags.maskCommand;
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x07, keyDown: false); // cmd-x up
//    event2?.flags = CGEventFlags.maskCommand;
    event2?.post(tap: CGEventTapLocation.cghidEventTap);

}





func copy() {


    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x08, keyDown: true); // cmd-c down
    event1?.flags = CGEventFlags.maskCommand;
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x08, keyDown: false); // cmd-c up
//    event2?.flags = CGEventFlags.maskCommand;
    event2?.post(tap: CGEventTapLocation.cghidEventTap);

}


func copystyle() {

    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x08, keyDown: true); // opt-cmd-c down
    event1?.flags = [CGEventFlags.maskCommand, CGEventFlags.maskAlternate];
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x08, keyDown: false); // opt-cmd-c up
    //    event2?.flags = CGEventFlags.maskCommand;
    event2?.post(tap: CGEventTapLocation.cghidEventTap);


}

func pastestyle() {

    let event1 = CGEvent(keyboardEventSource: nil, virtualKey: 0x07, keyDown: true); // opt-cmd-v down
    event1?.flags = [CGEventFlags.maskCommand, CGEventFlags.maskAlternate];
    event1?.post(tap: CGEventTapLocation.cghidEventTap);

    let event2 = CGEvent(keyboardEventSource: nil, virtualKey: 0x07, keyDown: false); // opt-cmd-v up
    //    event2?.flags = CGEventFlags.maskCommand;
    event2?.post(tap: CGEventTapLocation.cghidEventTap);

}

When giving an answer it is preferable to give [some explanation as to WHY your answer](http://stackoverflow.com/help/how-to-answer) is the one. — Stephen Rauch, Feb 10 '17 at 02:06

score -4 · Answer 3 · answered Oct 18 '16 at 11:40

-4

I'm pretty sure you can do this with Apple Events, but don't forget to set up a temporary exception entitlement to be able to message another app.

answered Oct 18 '16 at 11:40

Bob

910
1
8
12

What is "Apple Events"? I would appreciate a little more detail, if that's possible, please. Also, I do need to submit the app to the AppStore, if that detail is of any relevance. – Neovibrant Oct 18 '16 at 11:42

Swift macOS: How to paste into another application

3 Answers3

Linked