24

Is it possible for adb shell console to access an Android app's specific button using its id? or text?

I'm trying to automate the button click on the device. This is a web app accessed from a browser. So if I have that button id can I send an action to that button?

Alex P.
  • 30,437
  • 17
  • 118
  • 169
Siddharthan Asokan
  • 4,321
  • 11
  • 44
  • 80

5 Answers5

36

UI automator gives resource IDs, text, and the bounds of the UI element. An XML viewer or Chrome browser can be used to get a better view of the file.

adb pull $(adb shell uiautomator dump | grep -oP '[^ ]+.xml') /tmp/view.xml

The UI element's bounds can be be extracted and the mid point can be calculated. Replace text= with resource-id= or content-desc= as needed.

coords=$(perl -ne 'printf "%d %d\n", ($1+$3)/2, ($2+$4)/2 if /text="MY_BUTTON_TEXT"[^>]*bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"/' /tmp/view.xml)

Now we have the coordinates of the UI element's center in $coords and we just need to send a tap event.

adb shell input tap $coords
Jimmy Schementi
  • 1,239
  • 8
  • 17
apricot
  • 3,599
  • 3
  • 19
  • 20
  • 2
    This is way far a better answer, providing exactly the way to get bounds of an UI element vía adb. For some reason the `($1+$2)/2, ($3+$4)/2` is not working as expected. I've just fixed using only `$1, $3` because the very first pixel is good enough to be clicked :-) – m3nda Jun 19 '18 at 03:41
  • 3
    This is great and this is the one I picked. However... I decided to take it a bit further and get rid of this temporary file. When dumping, it is possible to specify the file to which one wants to dump, then I digged a bit more to find a device that represents the stdout. It soon became: `adb shell -x uiautomator dump /dev/fd/1` The -x option was necessary in order to have adb shell's output redirected to our main shell – Vinicius Dantas Jul 24 '18 at 02:25
  • can you please write it in batch (windows) – Amin Guermazi Mar 23 '21 at 11:01
  • For cmd: `for /f "tokens=1,2 delims= " %X in ('perl -ne "printf '%d %d', ($1+$3)/2, ($2+$4)/2 if /text=\042RUN\042[^>]*bounds=\042[(\d+),(\d+)][(\d+),(\d+)]\042/" window_dump.xml') do echo %X %Y` – Sunfarm Mar 24 '21 at 01:13
13

Yes you can, although it's not pretty.

You can get the view hierarchy by using this command:

adb exec-out uiautomator dump /dev/tty, which prints the output to the screen as an XML file. Unfortunately it's not valid XML, as there's some text appended. So let's filter that out:

adb exec-out uiautomator dump /dev/tty | awk '{gsub("UI hierchary dumped to: /dev/tty", "");print}'

Now we've got a valid XML. Let's run this through XPath:

adb exec-out uiautomator dump /dev/tty | awk '{gsub("UI hierchary dumped to: /dev/tty", "");print}' | xpath '//node[@resource-id="com.example:id/btnDone"]' 2> /dev/null, which will search for all nodes with the specific ID. Now you got the node, and you can do some more stuff on this.

If you were to get only the viewbounds, you'd have to do this:

adb exec-out uiautomator dump /dev/tty | awk '{gsub("UI hierchary dumped to: /dev/tty", "");print}' | xpath '//node[@resource-id="com.example:id/btnDone"]/@bounds' 2> /dev/null | awk '{$1=$1};1' | awk '{gsub("bounds=", "");print}' | awk '{gsub("\"", "");print}', which cleans up the string afterwards and just outputs [294,1877][785,1981].

The source of the specific node is:

<node index="1" text="Let’s get started!" resource-id="com.example:id/btnDone" class="android.widget.Button" package="com.example" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[294,1877][785,1981]" />

Evin1_
  • 12,292
  • 9
  • 45
  • 47
Niels
  • 9,264
  • 4
  • 29
  • 26
  • 1
    We could avoid using the final `awk` calls if we use the xpath's `string()` method: ```$ adb exec-out uiautomator dump /dev/tty | awk '{gsub("UI hierchary dumped to: /dev/tty", "");print}' | xpath 'string(//node[@resource-id="REPLACE_YOUR_ID_HERE"]/@bounds)' 2> /dev/null ``` – Evin1_ Aug 23 '19 at 19:49
9

I think the best you can do is to inject touch based on a coordinate.

Please see send touch event from ADB to a device and simulating touch using ADB

You might get a coordinate of a button from dumpsys window or activity.

You could also check out Monkeyrunner.

Community
  • 1
  • 1
pt2121
  • 11,720
  • 8
  • 52
  • 69
6

You won't be able to use the button id, but you can get the button coordinates and simulate a tap event.

Android comes with an input command-line tool that can simulate miscellaneous input events. To simulate tapping, use:

input tap x y

You can use the adb shell to run the command remotely:

adb shell input tap x y

Other options are:

shell@m0:/ $ input
input
usage: input ...
       input text <string>
       input keyevent <key code number or name>
       input [touchscreen|touchpad|touchnavigation] tap <x> <y>
       input [touchscreen|touchpad|touchnavigation] swipe <x1> <y1> <x2> <y2> [duration(ms)]
       input trackball press
       input trackball roll <dx> <dy>
Pedro Lobito
  • 94,083
  • 31
  • 258
  • 268
  • Is it possible to get the button coordinates in an automated, scripted way (so that it will run on different devices/screens)? – friederbluemle Mar 01 '16 at 14:02
  • 1
    I'd use the UI automator and click the button by its id or any other unique identifier. http://developer.android.com/tools/help/uiautomator/index.html – Pedro Lobito Mar 01 '16 at 14:17
  • Thanks, do you know of any tools that run directly in the shell? – friederbluemle Mar 01 '16 at 14:26
  • You may want to try monkey runner, here's 5 tools for automation, including monkey runner : http://www.softwaretestinghelp.com/5-best-automation-tools-for-testing-android-applications/ – Pedro Lobito Mar 01 '16 at 14:27
1

if me, i use python to do that, combine cmd command with subprocess.Popen, the script would be:

from subprocess import Popen, PIPE

import os

adbpath = r'YOUR ADB DIR'
adbfile = 'adb.exe'
adb = os.path.join(adbpath, adbfile)

def tap(a):
    cor = a
    t = Popen([adb, 'shell', 'input', 'tap', str(a[0]), str(a[1])])

#i give you extra script

def swipe(a, b):
     first = a
     last = b
     sab = Popen([adb, 'shell', 'input', 'swipe', str(first[0]), str(first[1]), str(last[0]), str(last[1]), '200'])

then when you have a button coord to tap just do this:

ok_coord = 122,137
tap(ok_coord)

when you want to swipe:

coord1 = 122,373
coord2 = 122,26
swipe(coord1, coord2)

to get the coordinate of button or a point, use 'uiautomatorviewer.bat' that exists in folder 'sdk\tools\bin'

or you can use apricot, erm3nda or Vinicius Dantas method,

Wahyu Bram
  • 413
  • 1
  • 7
  • 13