MODIFY the view hierarchy of another app with an Accessibility Service

Question

It is common knowledge that we can observe or query the view hierarchy of any app with an AccessibilityService:

Create your own accessibility service.

It is also possible to perform actions on behalf of the user:

Developing an Accessibility Service for Android.

My question is, can we modify the view hierarchy of a foreground app with an AccessibilityService?

I have already referred the following questions:

What they're doing is using the WindowManager and the SYSTEM_ALERT_WINDOW permission to overlay a view on top of the app in the foreground. The problem with this approach is that if the user presses BACK or HOME, the app is dismissed, but the view remains visible on the screen, even after the app is gone. The view is on TOP of the view hierarchy, and not a part of it.

Is there a way to add / modify the AccessibilityNodeInfo objects?
Are these the same as a View or ViewGroup?
Are we allowed to add views or modify existing views with an AccessibilityService?

My requirement is to display a small view within the app itself. It has to be part of the view hierarchy of the app, so that it stays or goes with the app. For this I need to modify the view hierarchy of the app, namely the AccessibilityNodeInfo objects retrieved from the service.

What I want is something similar to addView(), but add the View to the view hierarchy of the app itself, not on top of it.

How can we do this? Is this possible?

UPDATE:

Apps that support Custom Views for Accessibility

Can't you hide the overlay when the app you want to draw over leaves the foreground? — ataulm, Dec 11 '19 at 23:29

score 1 · Answer 1 · answered Dec 11 '19 at 23:27

1

No, you can't modify the view hierarchy of another app because it exists in a separate process.

This is similar to not being able to modify accessibility nodes from within an accessibility service.

answered Dec 11 '19 at 23:27

ataulm

15,195
7
50
92

score 1 · Answer 2 · answered Dec 13 '19 at 08:38

1

1) You can exploit draw over other apps permission. That solution will allow you only to draw overlays over another apps and not to change another apps behavior.

2) You can exploit instrumentation test mechanism. If you have enough information about the app(app id, activity name) and enough privileges (Run an instrument test from within app and wait for result), or root privileges. Here is an example:

@RunWith(AndroidJUnit4::class)
class InjectView {
    @get:Rule
    val activityRule = ActivityTestRule<MainActivity>(MainActivity::class.java)

    @Test
    fun injectView() {
        val rootLayout = activityRule.activity.findViewById<ViewGroup>(android.R.id.content)
        activityRule.runOnUiThread {
            rootLayout.addView(TextView(activityRule.activity).apply {
                text = "Injected View"
            })
        }

        Thread.sleep(10_000)
    }
}

answered Dec 13 '19 at 08:38

Link182

733
6
15

Thank you for taking the time to answer. The first one I know of, but the second one is new to me. Is this actually doable? As I understand it, the second approach requires signature level permission, which we likely won't have. – Yash Sampat Dec 13 '19 at 12:51
Also, the problem with the first approach (which I'm already using btw) is that it isn't within the view hierarchy of the app. This creates a problem when the foreground views change, the user navigates elsewhere or the app is dismissed. – Yash Sampat Dec 13 '19 at 12:54
Is not a option to use the second solution without enough privileges. Do you have root access or any another possibility to access another apps files? Maybe to inject the logic directly in dex files from storage? – Link182 Dec 13 '19 at 16:36

score 1 · Answer 3 · answered Dec 18 '19 at 04:14

You can still go with the approach of drawing on top of the app, not as part of its view hierarchy (which is impossible) - using SYSTEM_ALERT_WINDOW permission.

In order to know when the app is dismissed and dismiss your own overlay - listen to the accessibility event TYPE_WINDOW_STATE_CHANGED and check that there's a package change.

You can also go further with listening to TYPE_WINDOW_CONTENT_CHANGED and determining that there was some layout update. More accessibility events might come in handy or fine-tuning your overlay's accuracy.

In short - as long as you have the appropriate information to know about the underlying app's layout and when things happen, you can draw on top as if it's part of the app. Might be tricky and require some calculations since you're not able to just push views into the hierarchy, but totally doable.

Thank you for your feedback. I am already using SYSTEM_ALERT_WINDOW permission and showing an overlay. Will attempt to satisfy my requirements with the approach you have suggested and get back to you ... :) — Yash Sampat, Dec 24 '19 at 04:22

MODIFY the view hierarchy of another app with an Accessibility Service

3 Answers3