jenkins uiautomator build always green

Question

So I defined many UiAutomatorTestCase classes, each has 1 or 2 test cases at the most. Then I use Shell script on Jenkins to string these test cases into a series of tests, for example:

adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass1
adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass2
adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass3
adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass4
...
so on so forth.

one of the two (1/2) problems I have is that with Jenkins builds, it doesn't matter if any of these test fails, Jenkins always shows up as green, I need Jenkins stop and shows red for the build.

the other (2/2) problem is that if the app crashes in one of the tests, say, TestClass2, the script will try to pick up and continue executing. What would be the best method to make the script stop?

Any suggestions? Thanks

Arzath · Accepted Answer · 2013-08-09T20:09:54.873

Thanks to @KeepCalmAndCarryOn and Daniel Beck, I solved my problem by 2 steps:

log adb output
grep the keyword from the log

the code in the shell script would be:

#!bin/bash

function punch() {
  "$@" | tee ${WORKSPACE}/adb_output.log
  [[ -z "$(grep 'FAILURES!!!' ${WORKSPACE}/adb_output.log)" ]] || exit 1
}

punch adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass1
punch adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass2
...

score 1 · Answer 2 · edited May 23 '17 at 11:58

1

You need to check the exit code of every adb statement you are running before moving onto the next

This post here details checking exit codes with bash Checking Bash exit status of several commands efficiently

which includes this (changed for your example)

function test {
    "$@"
    status=$?
    if [ $status -ne 0 ]; then
        echo "error with $1"
        exit "$status"
    fi
    return $status
}

test adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass1
test adb shell uiautomator runtest myTest.jar -c com.myTest.TestClass2

You can also tell Jenkins to use bash by adding a shebang to the first line of your script

#!/bin/bash

edited May 23 '17 at 11:58

Community

1
1

answered Aug 07 '13 at 23:17

KeepCalmAndCarryOn

8,817
2
32
47

@KeepCalAndCarryOn Thanks for your reply, but I get FAILURES!!! Tests run: 1, Failures: 0, Errors: 1 INSTRUMENTATION_STATUS_CODE: -1 + s=0 + '[' 0 -ne 0 ']' + return 0 I took out the exit "$status" inside of the if clause, too. Perhaps $1 in this context is adb alone (which explains return 0)? – Arzath Aug 08 '13 at 19:18

jenkins uiautomator build always green

2 Answers2