7

I'm writing a AccessibilityService and I want to create view overlays on the views from the current activity that the accessibility service can retrieve. I have no problems to retrieve all AccessibilityNodeInfo objects from the current activity, but I have no idea how to get the views from these objects to create overlays. Unfortunately there are only few examples regarding accessibility services. Maybe some of you already have experience with this topic. I hope you can help me! Thanks!

EDIT: A paper shows that overlays over an activity's view contents are possible:

The display overlay is able to perform these tasks thanks to the Android Accessibility Framework [10]. Using the accessibility API, it is able to access and inspect the GUI layout of the applications on the screen, without requiring modifications or the instrumentation of the application code."*

Link: http://www.onarlioglu.com/publications/fc2015babelcrypt.pdf

Page 6 and 8. Thanks!

Steve Blackwell
  • 5,904
  • 32
  • 49
zuuurek
  • 81
  • 1
  • 3
  • 2
    Retrieve the on-screen bounds from the `AccessibilityNodeInfo`. Convert to coordinates that are relative to your overlay's on-screen bounds. Draw to your overlay. – alanv Mar 08 '15 at 08:19

2 Answers2

4

You cannot get the View objects from other apps, as the View objects are in a separate process from yours.

CommonsWare
  • 986,068
  • 189
  • 2,389
  • 2,491
  • Thank you for the fast answer! But them how is it possible to create view overlays on the view objects? I know that this has to be possible, because I read a paper about it. The GUI of the activity has been traversed via the Accessibility Service and then view overlays has been created. – zuuurek Feb 15 '15 at 18:56
  • @zuuurek: "But them how is it possible to create view overlays on the view objects?" -- I have no idea what you mean by this. "I know that this has to be possible, because I read a paper about it" -- do you have a link? – CommonsWare Feb 15 '15 at 18:58
2

Drawing overlays over the window using accessibility services is easy. I know this is an old question, but I thought I'd add the answer anyway.

RelativeLayout relativeLayout = new RelativeLayout(getContext());

WindowManager.LayoutParams topButtonParams = new WindowManager.LayoutParams(
    width, //The width of the screen
    height, //The height of the screen
    WindowManager.LayoutParams.TYPE_SYSTEM_OVERLAY,
    WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,
    PixelFormat.TRANSLUCENT);

//Just trust me, this alpha thing is important.  I know it's weird on a "translucent" view.
topButtonParams.alpha = 100;

relativeLayout.setLayoutParams(topButtonParams);

mWindowManager.addView(relativeLayout, topButtonParams);

Also, you need this permission in your manifest

<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />

Once you have an invisible relative layout overlay over the top of the entire screen, adding views to it is easy!

MobA11y
  • 18,425
  • 3
  • 49
  • 76