3

I am looking some way where I can listen to the events like a click inside certain application. I am trying to develop an application(electron) where I have to generate some specific triggers when certain thing happens. For example, my application is watching the browsers and when the user opens their gmail account and views some emails then my application knows about it and then present the users with some helpful information based on the contents of emails. Also the application must be able to view these type of events in different application like Slack, Outlook, Safari etc.

I heard about a feature called Accessibility but it seems there is very less documentation on implementing it into the code. Is there any way of achieving this functionality or is it feasible at all? It would be very helpful if someone can answer this question and provide the necessary links to the documentation. Thanks.

slugolicious
  • 15,824
  • 2
  • 29
  • 43
Shak
  • 134
  • 3
  • For email, you can embed a image for example in it, and when the email get read, the image linked to your website, and your web page get loaded and you can from there to know the email is get read. – Tony Dong Jan 31 '19 at 19:00
  • You need to write native code to be able to register your application as an Accessibility Service. Documentations regarding this is available here: https://developer.apple.com/documentation/applicationservices/axuielement_h#//apple_ref/c/func/AXIsProcessTrustedWithOptions The simplest example of a real application doing so is available here: https://github.com/aahung/Unshaky For the record, I would not allow access to any program asking for accessibility permission. This is a very bad road you are trying to go. Try what @customcommander said in his answer. – Soroush Falahati Feb 01 '19 at 01:51

1 Answers1

3

This should have been a comment rather than an answer but it is too long for a single comment.

Couple of comments:

  1. The Accessibility feature isn't probably what you think it is. It's about how you can make your application usable by sight-impaired users for example. See this guide.

  2. It would seem odd (dangerous even) to be able to have so much control over other applications without involving some sort of authorisation and/or integration with some third-party providers beforehand.

  3. It may be possible that your package, in addition to your app, comes with some plugins. Upon installation of your package, you may be able to detect the presence of Slack for example, and trigger the process of registering your plugin with the user-installed instance of Slack. (If Slack allows the installation of third-party plugins in this way; I don't know)

  4. In order to inspect the content of a browser, your best bet IMHO would be to write a browser extension that can talk to your app. This may actually be possible, see this post on Stack Overflow. Upon installation of your app, you could notify your users that your browser extension is available.

Unfortunately this is probably not the answer you're looking for but I hope it gives some valuable insights into what's doable and what isn't.

customcommander
  • 17,580
  • 5
  • 58
  • 84
  • He is talking about `accessibility services` in mac and the fact that you can write a program to be registered as an accessibility service and therefore retrieve information about what the user does. – Soroush Falahati Feb 01 '19 at 01:36
  • Thanks @SoroushFalahati for your clarification. That’s probably what the OP meant indeed. – customcommander Feb 01 '19 at 06:27