1

I'm currently attempting to make an AI that will be able to learn how to play games on Android by actively monitoring certain features pertaining to the pixels on the screen, and how the user interacts with them.

My issue is that I cannot seem to find any relevant information regarding detecting MotionEvents that other applications receive. Are there any standard means by which I could set a global OnTouchEvent hook, thus receiving all user inputs regardless of the application that is active? If there aren't any standard methods, any ideas as to how one could achieve this?

Shane Duffy
  • 1,117
  • 8
  • 18

2 Answers2

4

One application can not know anything that is going on inside another application unless that other app has specifically shared that data. This is for security and privacy reasons. (Imagine how unsafe phones would be if any app could know what the user is entering into any other app, such as password.)

Doug Stevenson
  • 297,357
  • 32
  • 422
  • 441
  • I refuse to accept this. There MUST be some means by which this can be achieved. I'm thinking I can create a transparent system overlay which ascends all windows, allowing me to capture click events, at which point I can remove the overlay to send the actual click to the active application. Is there any reason why this wouldn't work? – Shane Duffy Mar 05 '16 at 08:02
  • No, you can't do this. You can no longer do overlays in Android L. Before L, you could do an overlay, but that would trap all the events before they reached the app beneath it. It is considered a security problem. You can't simply bypass the security constraints of Android. – Doug Stevenson Mar 05 '16 at 08:05
  • You cannot do overlays? Where is this specified? – Shane Duffy Mar 05 '16 at 08:16
  • 2
    You can read about it here (sorry, it was M, not L when this was changed): http://www.codeproject.com/Tips/1056871/Android-Marshmallow-Overlay-Permission Even if the user gives permission, that doesn't mean you can invisibly trap events. If you cover the whole screen, all the events will go to your app and not the app underneath. This will defeat the purpose of your experiment. – Doug Stevenson Mar 05 '16 at 08:23
  • 1
    Also see this: http://stackoverflow.com/questions/9656185/type-system-overlay-in-ics – Doug Stevenson Mar 05 '16 at 08:26
  • Hrm, I presume it is also impossible to send click events to external apps as well, so yeah I guess my plight has ended :P Thanks though! – Shane Duffy Mar 05 '16 at 08:48
  • Yeah, sending to arbitrary apps would be just as bad, if not worse, than detecting. Sorry about that. – Doug Stevenson Mar 05 '16 at 08:55
3

You can detect external touch event in your application. WINDOWS_SERVICE which is system service, can provide you with the information of touch on the Android device, for that you need one layout which should be of unit size of pixel so that it doesn't consumes you click event on external applications. 

mWindowManager = (WindowManager) getSystemService(WINDOW_SERVICE);

WindowManager.LayoutParams params = new WindowManager.LayoutParams(
1, // width is equal to 1px
1, // height is equal to 1px
WindowManager.LayoutParams.TYPE_PHONE, // Type Phone, These are non-application windows providing user interaction with the phone
WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE | // This window would never get key input focus.
WindowManager.LayoutParams.FLAG_WATCH_OUTSIDE_TOUCH, // This window will get outside touch.
PixelFormat.TRANSPARENT // The view will be transparent
);

Now you have to add you layout to this window manager service.

//Adding view to the window manager
mWindowManager.addView(touchLayout, params);

For better explanation with code you can refer this link http://allinmyspace.com/2016/08/28/android-detect-touch-events-on-external-applications/

Sachin Chauhan
  • 356
  • 1
  • 11
  • Link is now broken, any chance for an update? (Been a bit of a while, I know, but worth a shot) – Yuval.R Mar 16 '23 at 09:17