How to make Jenkins display fail if one of the ant tests fail

Question

I have several tests that should be executed regardless of each other's success and I want Jenkins/Hudson to display red light if at least one of those tests failed. My current (simplified for clarity) configuration is as following:

ci.sh:

...
ant
...

build.xml:

...        
<target name="AllTests">
    <antcall target="TestA"/>
    <antcall target="TestB"/>
    <antcall target="TestC"/>
</target>

<target name="TestA">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestB">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestC">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>
...

How can I make all the tests to execute anyways, but ant/Jenkins should fail if at least one of the three failed?

Mahdi Yusuf · Answer 1 · 2011-07-14T14:13:44.357

2

I would set a variable in your ant script, and have the ant script exit with that variable if the test fails. Jenkins jobs fail, if the exit code is anything but zero. Like below:

#!/bin/bash
zero="0"

*** run your ant script ***

if [$? -gt $zero]; # check to see if exit code of ant script is great than zero
then
    exit(5) # non-zero exit means failure.
else
    echo Success
    exit(0)
fi

edited Jul 14 '11 at 14:13

answered Jul 13 '11 at 20:05

Mahdi Yusuf

19,931
26
72
101

I've just found a viable workaround, but your answer seems to be more correct and generic. Would it be possible to ask for a code snippet, and I'd accept your answer; mine will be there just as a supplementary? – BreakPhreak Jul 14 '11 at 07:46
I am not familiar with ant scripts so I will write in bash; it will get the point across. – Mahdi Yusuf Jul 14 '11 at 13:59
Thanks, but we've missed the point here. It's about how `ant` should support it. Three targets, one fails, all the task fails but all the three targets should be executed. Thanks anyways. – BreakPhreak Jul 14 '11 at 15:13
you said you wanted to set build to red if any of the builds failed. I told how to? you alternately did that by limiting it to one failing thread as supposed to how many were running before. My solution does answer you question. It will help others. If it didn't help you its all good. :) – Mahdi Yusuf Jul 14 '11 at 19:55
Sure it will help others and many thanks for that. I am just trying to follow the mainstream of ant as a primary option. – BreakPhreak Jul 17 '11 at 07:29

score 1 · Accepted Answer · answered Jul 14 '11 at 07:45

I have found parallel task with thread count set to "1" as a viable workaround. This isn't perfect, but the change in build.xml is minimal:

build.xml:

...        
<target name="AllTests">
    <parallel threadCount="1" timeout="900000">
        <antcall target="TestA"/>
        <antcall target="TestB"/>
        <antcall target="TestC"/>
    </parallel>
</target>

<target name="TestA">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestB">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestC">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>
...

score 0 · Answer 3 · answered May 08 '12 at 20:02

Huh? AFAIK, ant per default does not stop on test failures or test errors. And Hudson/Jenkins can be told to collect JUnit test reports afterwards and alter the build status depending on the test results... (there is at least one plugin for Jenkins that allows you to set watermarks)

See http://ant.apache.org/manual/Tasks/junit.html - haltonfailure and haltonerror are off by default.

score 0 · Answer 4 · answered Sep 27 '12 at 09:33

This is sequential ant solution, but it changes your build.xml a little more. It uses a temporary file to flag that one test failed.

<?xml version="1.0" encoding="UTF-8"?>
<project name="project" default="AllTests">
    <property name="test-failed.file.path" location="test.failed" />

    <macrodef name="checkTestResult">
        <sequential>
            <condition property="allOK">
                <equals arg1="${test.result}" arg2="0" />
            </condition>
            <antcall target="-create-test-failed-file" />
        </sequential>
    </macrodef>

    <target name="-create-test-failed-file" unless="${allOK}">
        <touch file="${test-failed.file.path}" />
    </target>

    <target name="AllTests">
        <delete file="${test-failed.file.path}" />

        <antcall target="TestA" />
        <antcall target="TestB" />
        <antcall target="TestC" />

        <available file="${test-failed.file.path}" property="oneTestFailed" />
        <delete file="${test-failed.file.path}" />
        <fail if="${oneTestFailed}" message="At least one test failed" />
    </target>

    <target name="TestA">
        <exec executable="hostname"
              failonerror="false"
              resultproperty="test.result" />
        <checkTestResult />
    </target>

    <target name="TestB">
        <!-- This one fails -->
        <exec executable="hostname"
              failonerror="false"
              resultproperty="test.result">
            <arg value="--NoSuchOption" />
        </exec>
        <checkTestResult />
    </target>

    <target name="TestC">
        <exec executable="hostname"
              failonerror="false"
              resultproperty="test.result" />
        <checkTestResult />
    </target>
</project>

score 0 · Answer 5 · edited May 23 '17 at 10:29

You can install Log Parser Plugin to fail the build on certain pattern in the console log.

So all the scripts may run, but the build may fail at the end when errors/warnings appear in the log section (which you can configure to be printed).

The rules can be configured in the Parsing Rules File, like:

# match line starting with 'error ', case-insensitive
error /(?i)^error /

# list of warnings here...
warning /[Ww]arning/
warning /WARNING/

In case you've 3 scripts and Jenkins stops build on the 1st one, you can add #!/bin/sh -x to the Execute shell to not stop on the first error and use above mentioned Log Parser plugin to fail the build at the end.

See also: How/When does Execute Shell mark a build as failure in Jenkins?

score 0 · Answer 6 · answered Feb 23 '17 at 15:29

0

Just adding my point: If you want to manually fail your ANT task and want your jenkins job to fail.

Then you can use something like this

<fail status="-1" message="" />

jenkins jobs fails based on the errorcode returned by Ant task.

answered Feb 23 '17 at 15:29

Abdul Kader

5,781
4
22
40

How to make Jenkins display fail if one of the ant tests fail

6 Answers6

Linked