android accessibility : Get all EditText node in a webview

Question

I am trying to create a password app manager. I want to create an Autofill feature. I saw Android autofill, but it doesnt work perfectly on browser app. So I decided to use the accessiblity to get all editText.

If I find 2 EditText including one in password mode, I enable autofill. (I suppose it is a login form)

When I loop into all nodes in the view, I will only find the EditText focused.it's like this I couldn't find all the node.

Is there a way to find all the node of the screen for a browser app?

Here is the code

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
 
    switch(event.getEventType()) {
        case AccessibilityEvent.TYPE_VIEW_CLICKED:
        case AccessibilityEvent.TYPE_VIEW_FOCUSED:
            try {
                if(event.getClassName() == null) return;
                Log.e(TAG, "onAccessibilityEvent: " + getEventType(event)   );



                Class className = Class.forName(event.getClassName().toString());

                if (EditText.class.isAssignableFrom(className)) {

                    /** parse ALL NODE -> see  getCredFieldsFromViews  **/
                    Log.i(TAG + "-info", AccessiblityParser.getInfoFromEvent(getRootInActiveWindow()).toString());


                }
            } catch (ClassNotFoundException e) {
                e.printStackTrace();
            }
            break;
    }

}


 

@Override
public void onServiceConnected() {
 
    AccessibilityServiceInfo info = getServiceInfo();
    info.eventTypes = AccessibilityEvent.TYPES_ALL_MASK;

 
    info.feedbackType = AccessibilityServiceInfo.FEEDBACK_ALL_MASK; //.CAPABILITY_CAN_RETRIEVE_WINDOW_CONTENT;

    info.flags = AccessibilityServiceInfo.DEFAULT | AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS |
            AccessibilityServiceInfo.FLAG_REQUEST_TOUCH_EXPLORATION_MODE | AccessibilityServiceInfo.FLAG_RETRIEVE_INTERACTIVE_WINDOWS
            |  AccessibilityServiceInfo.FLAG_REPORT_VIEW_IDS;
 


    //info.flags = AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS ;
    info.notificationTimeout = 100;

    this.setServiceInfo(info);

}

Loop all the nodes :

private static void getCredFieldsFromViews(AccessiblityParser accessiblityParser,AccessibilityNodeInfo info) throws ClassNotFoundException {

    if (info == null)
        return;

    if(info.isEditable()){
   
        if( info.isPassword() ){ // password EDIT TEXT
           accessiblityParser.mdpNode = info;
        }else{ // (maybe) username EDIT TEXT 
          accessiblityParser.usernameNode = info;
        }

    }

    int childCount = info.getChildCount();
    for(int i=0;i<childCount;i++){
        AccessibilityNodeInfo child = info.getChild(i);
        if(child != null)
        child.refresh();

        getCredFieldsFromViews(accessiblityParser,child);
        if(child != null){
            child.recycle();
       
        }
        test++;

    }
    return;
}

EDIT :

I saw Accessibility and Android WebView Maybe I can inject script if it is not deprecated ?

Another way maybe use :

arguments.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_HTML_ELEMENT_STRING, "INPUT");
            r = info.performAction(AccessibilityNodeInfo.ACTION_NEXT_HTML_ELEMENT, arguments);

But how to use it ?

Thank

android accessibility : Get all EditText node in a webview

0 Answers0