13

I'm executing some tests in my Android device by using the following command:

adb shell am instrument -w package.name/android.test.runner.AndroidJUnitRunner

I can see the test progress and simple results through STDOUT. But, does this process also generates a results file inside the device (xml, html, etc)? If yes, where is it stored?

Thanks

Allen Hair
  • 1,021
  • 2
  • 10
  • 21
Marcelo
  • 2,245
  • 4
  • 20
  • 24

4 Answers4

9

But, does this process also generates a results file inside the device (xml, html, etc)?

No, it does not.

Report generation is generally handled at a higher layer than 'am instrument'. If you run your tests using Gradle, it should generate the report for you. I believe this is what Android Studio relies on as well.

If you must generate the report from the test itself, you can use a custom test runner. See this answer for one way to do it: http://www.stackoverflow.com/a/5574418/1999084

Allen Hair
  • 1,021
  • 2
  • 10
  • 21
  • Is there any solution/approach to have the test results displayed in a better way then? I'm asking this because when I run the tests through Android Studio it generates the junit tests output in xml, html, etc. – Marcelo Nov 24 '15 at 19:43
  • 1
    Report generation is generally handled at a higher layer than 'am instrument'. If you run your tests using Gradle, it should generate the report for you. I believe this is what Android Studio relies on as well. – Allen Hair Nov 24 '15 at 19:59
  • 1
    If you must generate the report from the test itself, you can use a custom test runner. See this answer for one way to do it: http://stackoverflow.com/a/5574418/1999084 – Allen Hair Nov 24 '15 at 20:01
  • Thanks, it helped. Could you add those information in your main answer? – Marcelo Nov 24 '15 at 20:11
  • Just an additional information: If you are using the test runner that google page is recommending (AndroidJUnitRunner), probably the solution pointed out in this reply will not work. I found the solution in this great article: http://www.everybodytests.com/2015/04/jenkins-devices-androidjunitrunner.html – Marcelo Nov 26 '15 at 19:28
8

I had a similar problem (I wanted to have xml test reports for my Jenkins when it runs instrumentation tests on a device). I solved it by implementing the "android-xml-run-listener" (https://github.com/schroepf/TestLab/tree/master/android).

To use it simply add:

androidTestCompile 'de.schroepf:android-xml-run-listener:0.1.3'

to your build.gradle (note the androidTest prefix - this will not add code to your production app!).

To use it, add:

-e listener de.schroepf.androidxmlrunlistener.XmlRunListener

to your am instrument command.

And to retrieve the XML report file, use:

adb pull /storage/emulated/0/Android/data/<your-app-package-name>/files/report.xml
toby
  • 81
  • 1
  • 2
  • I am not able to see the report.xml generated on android emulator 5.1.1 device..Can you please help? – rupesh jain Feb 19 '18 at 23:56
  • hi @rupeshjain looks like on older android versions the `emulated/0` directory doesn't exist... For me I had to use `/storage/sdcard/Android/data/de.schroepf.demoapp/files` to access the reports on a 5.x emulator... – toby Mar 03 '18 at 07:57
3

If you are executing your tests using AndroidTestOrchestrator your XML test results are generated and stored inside the devices storage/emulated/0/odo/ directory. So they can be accessed using:

adb pull storage/emulated/0/odo/

I'm not sure why this isn't mentioned anywhere in the documentation. This path is likely to be different for real devices, where I believe the results are outputted on the SDCARD somewhere.

myles wilter-heritage
  • 31
  • 1
0

I'm in a similar situation and found https://docs.marathonlabs.io/ works great for my use case. Other options I've looked into:

  • https://docs.marathonlabs.io/ is an effort similar to spoon but seems to be more updated and better maintained. It also provides nice HTML reports with additional features like tests batching, retrying and screen recording.
  • https://github.com/square/spoon is a test runner wrapper on top of adb shell am instrument which first runs with -e log true to get a list of available Espresso tests, and then run each test case one-by-one with -r flag so that we can be getting the XML test result from the raw text output as well runtime at the per testcase level. I didn't go down this route because it makes our overall test execution time a lot longer.
  • Use https://github.com/jamesknowsbest/Instrumentationpretty to parse the output from instrumentation tests raw log (-r) flag. We will be able to generate an XML JUnit test result with all info, except for test execution time at the test case level.
  • Use adb shell am instrument -f <proto_file> to store the instrumentation result in a protobuf file, read it in and convert it to junit xml file. Looking at the Android instrument source code for am/instrument.java, it seems the testlevel runtime are kept when we run it with -f instead of -r. But this means we will have to create our own protobuf parser file to generate the Junit-style XML file yourself.
  • https://github.com/schroepf/TestLab/tree/master/android is another Open source tool created to add instrumentation data inside the application (as opposed to the client / runner side approach above). It should create a JUnit style XML result inside the Android emulator and we can then use adb pull to pull it out of the emulator afterwards.
user20429758
  • 1