0

Can onscreen objects can be recognized and identified programmatically in iOS and Android.

USE CASE

App 1 is open with its own active area, button, or other target object visible onscreen. Can App 2 recognize the contents onscreen and perhaps present a partial overlay button or perform its own task based on object's presence or lack thereof?

Is this possible on either iOS and/or Android, or are app's sessions protected on one or both OSes?

I found this article Understanding Object Identification in Android Open Applications but not only is a bit over my head, I don't think it's what I'm after.

I also found this Android: How can I get current opened application name on screen which is closer, but still doesn't answer my question.

This Gist seems to suggest that screen "states" can, in fact, be accessed, but can the objects on the screen be recognized and acted upon?

Community
  • 1
  • 1
KillerDesigner
  • 485
  • 1
  • 6
  • 18
  • On Android, this is not generally possible, for privacy and security reasons. Your first link is for a testing framework; test tools, driven from developer workstations, have access to some of this information, but other apps cannot. The closest thing on Android is through the accessibility APIs and the assist API. The latter comes closest to what you seek, but it requires user intervention (long-press HOME) to trigger. – CommonsWare Jan 03 '17 at 19:10
  • first of all you have to access both application (for example by API) otherwise for the security reason you can't – Moien.Dev Jan 03 '17 at 19:17
  • Not possible at all in iOS. – rmaddy Jan 03 '17 at 19:19
  • As rmaddy says, this is completely impossible on iOS (Jailbroken devices excluded) – Duncan C Jan 03 '17 at 22:56

0 Answers0