1

I have a django test suite which is running on jenkins with the following build:

virtualenv -p /usr/bin/python3.5 etl_api_virtualenv
cd etl_api_virtualenv
source bin/activate
python --version
cd $WORKSPACE/leg_apps
ls
pip install -r requirements-dev.pip
./api/test/run_api_tests.sh $WEB_ENV

This is working, but when some of the unit tests fail, the job/build still is marked as a success. I can't figure out how to make the job indicator turn red and mark as a failure if the tests don't all pass. I'm not even sure what to Google to try to figure this out, so please forgive the apparent "lack of effort"... it's not the case. I'm just dead-ended at the start.

EDIT:

I think the problem might be that I don't know how to make run_api_tests.sh return an appropriate code based on whether or not the tests fail

thumbtackthief
  • 6,093
  • 10
  • 41
  • 87
  • Possible duplicate of [Handling exit code returned by python in shell script](https://stackoverflow.com/questions/14259660/handling-exit-code-returned-by-python-in-shell-script) – YPCrumble Feb 08 '18 at 19:03

1 Answers1

2

You need to make sure that any non-zero exit codes are propagated to Jenkins. Jenkins is viewing the exit codes of the commands you run, so as long as your bash script makes it to the end without crashing or having a non-zero exit it will exit with code 0 signaling that nothing went wrong. Most likely you need to add something to the effect of the following to ./api/test/run_api_tests.sh:

<command_running_tests>
# Capture the most recent exit code using $?
if [ "$?" = "0" ]; then
    printf "[TEST] - executable built: ${EXEC}\n"
else
    printf "[TEST] - failed\n"
    exit 1 # or exit $? if you want the specific code
fi
LittleBoxOfSunshine
  • 174
  • 1
  • 9
  • Is the 0/1 automatic based on the success of the tests? I tried this code and it does not seem to have any effect. – thumbtackthief Feb 08 '18 at 20:15
  • Also if I just put the line `exit 1` by itself to automatically trigger, the job shows failure as expected. So I'm guessing it's something in the capture line that I'm not understanding. – thumbtackthief Feb 08 '18 at 20:19
  • It depends on how you are running your tests. Some test frameworks use non-zero exit codes when the tests fail to make it easy to test for that condition. If your setup isn't doing that you to replace the condition in the if statement with some other test based on the output of the test command. For example if it just prints "All tests passed." You can use `OUTPUT="$()"` and `if [ "${OUTPUT}" = "All tests passed."]; then` – LittleBoxOfSunshine Feb 08 '18 at 20:41