7

I have an Android test project that I'd like to link into Hudson, but I haven't found a way to output the test results as XML instead of text. Does anyone know if there's an easy way to do this already?

-Dan

Dan
  • 3,884
  • 3
  • 27
  • 32

5 Answers5

4

FYI, for anyone else who happens to stumble on this question. I've created a new TestRunner that you can use in your Android test projects that will output your test results in XML readable by Hudson (and probably any other CI app). You can read about it here: http://droiddudes.com/2010/04/07/athenatestrunner/ or just grab it from here: http://github.com/dwatling/athena

Dan
  • 3,884
  • 3
  • 27
  • 32
  • Thanks for opensourcing your tool. Unfortunately I have some problem with it. This is Athena output summary: Total tests: 22 Total failures: 0 (0.0) Total errors: 0 (0.0) Total time: 70 seconds but from eclipse I get: Runs 31/31, Errors: 3, Failures: 4. Could you help me with this? – Maciek Sawicki Mar 29 '11 at 11:44
  • In the output of Athena it displays the number of test suites found and the number of tests found in each suite. Could you check to see where the discrepancy is between the run inside of Eclipse and in Athena? Specifically, which suites and tests are not run in Athena, but are run in Eclipse? – Dan Mar 29 '11 at 17:45
  • Part of the count discrepancy is due to Eclipse reporting on (and running) the "testAndroidTestCaseSetupProperly" from AndroidTestCase. Athena does not run that test, it will only run the tests that are found in the *Test.java files. This is something that can be improved on in the future. – Dan Mar 29 '11 at 18:00
  • Thank you for fast answer. I switched to http://android-junit-xml-reporter.googlecode.com/ for now. – Maciek Sawicki Mar 30 '11 at 13:15
3

I should have added this answer awhile ago.

I've been using https://github.com/jsankey/android-junit-report for close to a year now and that has worked much better than Athena or anything else I've found.

Dan
  • 3,884
  • 3
  • 27
  • 32
1

We had similar problem in our company. We checked all the available open-source solutions and none of them was really perfect. So we developed and just open-sourced a solution for it. I still do not say an "ultimate" one but certainly much better than either athena or the python reporter or any after-test analysis. You can find it here: http://code.google.com/p/the-missing-android-xml-junit-test-runner/

It provides:

  • separate XML file per each package involved
  • XML files are generated on the device (need to be adb pull'ed after test)
  • timing of the tests is fully supported
  • we have full stack trace reported in failure/error case

Instead of analysing java source code (as in athena) or analysing the output (the python script), we extended android instrumentation runner. So we get all the benefits of using standard command line options for test selection, coverage enabling etc. - all described here: http://developer.android.com/guide/developing/testing/testing_otheride.html#RunTestsCommand.

We were able to successfully run the code using standard test rules with coverage analysed by emma, all nicely reported in Jenkins.

Jarek Potiuk
  • 19,317
  • 2
  • 60
  • 61
0

Nevermind. I found this question instead: How to Generate Android Testing Report in HTML Automatically

Community
  • 1
  • 1
Dan
  • 3,884
  • 3
  • 27
  • 32
0

As another follow-up, I've come up with a "decent" solution. When running my instrumentation tests, I had to use the '-r' option (e.g. adb shell am instrument -w -r com.myApp/android.test.InstrumentationTestRunner > tests-out.txt) and write my own parser to convert the output into XML format accepted by Hudson.

It isn't perfect as it doesn't provide timing, nor does it provide log output messages, but at least I now have Hudson reporting on my Android tests.

Dan
  • 3,884
  • 3
  • 27
  • 32