Don't fail the gradle build if a test is failing with the gradle-android-test-plugin

Question

I set my project up to run with Robolectric and the the gradle-android-test-plugin. This all works fine and first tests are running and failing.

If a test fails this will also fail the gradle build. Is there a way to just let the build go on and write down the failing tests for evaluation in a later step?

The plan is to integrate the testing in a continuous integration server and there the build should only be unstable if a test fails.

score 15 · Answer 1 · edited Oct 03 '16 at 15:50

15

Hmm. Well you have two options I think. One is to use

testTask.ignoreFailures = true

to not let the task fail when a test fails.

Another approach would be to run your gradle command with '--continue'. This will execute as many tasks as possible and list the failed tasks at the end and not stop after the first task has failed.

edited Oct 03 '16 at 15:50

kellyfj

6,586
12
45
66

answered Nov 22 '13 at 11:10

Rene Groeschke

27,999
10
69
78

There is no testTask in my project to change this value on. Maybe this is specific to the Java Testing tasks? I'm using Gradle for Android. – Janusz Dec 10 '13 at 13:57
"testTask" is just the placeholder for your test task. you have to set the ignoreTestFailures property to true on the task of type "Test" that is failing your build – Rene Groeschke Dec 11 '13 at 09:38
There is a task called test but setting this property is doing nothing. But I don't know if this is the correct test since the tests are executed by a third party plugin: android-test. – Janusz Dec 11 '13 at 10:11
Is this property suppose to be ignoreFailures = true ? – KevinO Mar 18 '14 at 18:45
1

@KevinO, your right. typo on my side. it should be 'ignoreFailures'. I've updated my answer – Rene Groeschke Mar 23 '14 at 21:11
This does not work with the gradle-android-test-plugin, does it? – Cristian May 09 '14 at 18:11
Do you mind elaborating the `testTask.ignoreFailures = true` sample? I.e. where one should put it? Is it in the `project.gradle.taskGraph.whenReady` block? – Toochka Oct 28 '15 at 09:03
Could not find property 'testDebug' on task – Shridutt Kothari Jun 29 '16 at 05:03

score 15 · Accepted Answer · edited May 03 '22 at 07:34

15

The correct syntax with AndroidConnectedTests is as following:

project.gradle.taskGraph.whenReady {
    connectedAndroidTest {
        ignoreFailures = true
    }
}

Or for newer Gradle-plugin (1.3.0 and later), try:

project.gradle.taskGraph.whenReady {
    connectedDebugAndroidTest {
        ignoreFailures = true
    }
}

But if you have Flavors (in newer Gradle versions):

android {

    // ...

    project.gradle.taskGraph.whenReady {
        android.productFlavors.all { flavor ->
            // Capitalize (as Gralde is case-sensitive).
            def flavorName = flavor.name.substring(0, 1).toUpperCase() + flavor.name.substring(1)

            // At last, configure.
            "connected${flavorName}DebugAndroidTest" {
                ignoreFailures = true
            }
        }
    }
}

Now the test task is not failing the build anymore, which ensures the coverage report is created, and you can pick up the failed tests with your build server to mark the build as unstable etc.

edited May 03 '22 at 07:34

Top-Master

7,611
5
39
71

answered Aug 30 '14 at 21:43

Janusz

187,060
113
301
369

1

If you're running task (e.g.) `connectedAndroidTestFoo` instead of `connectedAndroidTest` then second line of the script (see above) should be `connectedAndroidTestFoo {`. `Foo` is a `flavour` – Toochka Aug 20 '15 at 06:40
Since 1.3.0 gradle version, you should check this answer https://stackoverflow.com/questions/30100781/android-gradle-plugin-1-2-2-flag-ignorefailures-causes-error-in-task-connecteda – Paulo Pereira May 19 '18 at 14:52
It looks like this no longer works. "gradle.startParameter.continueOnFailure = true" has a similar effect – G. Blake Meike Sep 14 '20 at 23:26

Kosh · Answer 3 · 2022-05-03T11:42:26.283

0

late to this but you could do:

    testOptions {
        unitTests.all {
            ignoreFailures = true 
        }
    }

within android {} block

edited May 03 '22 at 11:42

answered Mar 12 '22 at 14:52

Kosh

6,140
3
36
67

Above does nothing for Android-Tests (tests running on device/ Emulator), and any failed android-test blocks build process. – Top-Master May 03 '22 at 07:50
As a side note; 99% of our tests are usually Android-tests, which can do unit-tests as well, hence Gradle's "`unitTests`" name is confusing, and should be something like "`javaTests`" instead (as only Java API is available, and calling any Android API causes error). – Top-Master May 03 '22 at 07:55

Don't fail the gradle build if a test is failing with the gradle-android-test-plugin

3 Answers3

Linked