Jacoco and Unit Tests Code Coverage with android-gradle-plugin >= 1.1

Question

I recently started integrating android-gradle-plugin 1.1.0 in one of my projects. The project uses robolectric 2.4 to run unit tests.

It's a multi module project with very complex dependencies (Some modules depend on other modules). Something like that:

--> application-module (dependsOn: module1, module2, module-core)
    --> module1 (dependsOn: module-core)
    --> module2 (dependsOn: module-core)
    --> module-core (dependsOn: module3, module4)
        --> module3 (library dependencies)
        --> module4 (library dependencies)

For a more cleared picture please see jacoco-example project.

I tried to integrate JaCoCo to generate reports for the unit tests, but it seems to me that it runs only androidTests which are basically instrumentation tests.

After some google'ing I've come across a few projects on GitHub and other articles, but they mainly are focused on previous versions of the android-gradle-plugin or are using other third party plugins like android-unit-test for example here.

May be I've lost my ability to google. But can somebody point me in a direction where I can find some documentations regarding the new stuff in android gradle plugin and how to run the jacoco task only for unit tests?

UPDATE

Adopted the script from nenick's example:

apply plugin: "jacoco"

configurations {
    jacocoReport
}

task jacocoReport(dependsOn: 'testDebug') << {
    ant {
        taskdef(name:'jacocoreport',
                classname: 'org.jacoco.ant.ReportTask',
                classpath: configurations.jacocoReport.asPath)

        mkdir dir: "${buildDir}/test-coverage-report"
        mkdir dir: "${buildDir}/reports/jacoco/test/"

        jacocoreport {
            executiondata = files("${buildDir}/jacoco/testDebug.exec")

            structure(name: "${rootProject.name}") {
                classfiles {
                    fileset (dir: "${buildDir}/intermediates/classes/debug") {
                        //exclude(name: '**/*_*.class')
                        exclude(name: '**/R.class')
                        exclude(name: '**/R$*.class')
                        exclude(name: '**/BuildConfig.class')
                    }
                }

                sourcefiles {
                    fileset dir: "src/main/java"
                    fileset dir: "${buildDir}/generated/source/buildConfig/debug"
                    fileset dir: "${buildDir}/generated/source/r/debug"
                }
            }

            xml destfile: "${buildDir}/reports/jacoco/test/jacocoTestReport.xml"
            html destdir: "${buildDir}/test-coverage-report/"
        }
    }
}

dependencies {
    jacocoReport 'org.jacoco:org.jacoco.ant:0.7.2.201409121644'
}

After that the ./gradlew jacocoReport executes and generates the report, but it shows 0 (zero) test coverage, which is impossible because at least half of all classes are tested.

UPDATE_2

Tried out this example. Adding the next task to one of my gradle build files:

task jacocoTestReport(type:JacocoReport, dependsOn: "testDebug") {
    group = "Reporting"
    description = "Generate Jacoco coverage reports"

    classDirectories = fileTree(
            dir: "${buildDir}/intermediates/classes/debug",
            excludes: ['**/R.class',
                       '**/R$*.class',
                       '**/*$ViewInjector*.*',
                       '**/BuildConfig.*',
                       '**/Manifest*.*']
    )

    sourceDirectories = files("${buildDir.parent}/src/main/java")
    additionalSourceDirs = files([
            "${buildDir}/generated/source/buildConfig/debug",
            "${buildDir}/generated/source/r/debug"
    ])
    executionData = files("${buildDir}/jacoco/testDebug.exec")

    reports {
        xml.enabled = true
        html.enabled = true
    }
}

Same issue, the reports are generated, but the code coverage is still zero.

UPDATE_3

It seams that the task from UPDATE_2 worked but only for the module with apply plugin: 'com.android.application' (The reports a generated correctly). But for modules that are android libraries (apply plugin: 'com.android.library') the reports show zero coverage, although the modules contain more tests then the application module.

UPDATE_4

Created a simple example project that demonstrates my issue. Currently if you run ./gradlew jacocoReport the report is generated, but no test coverage is displayed for the module projects. See this link

Short note: When the tests were AndroidUnitTests (whiteout JUnit 4 and Robolectric) JaCoCo reports showed coverage for all the modules.

Any ideas?

I had something like this working (unreliably) with 1.1-rc1. Now I'm running 1.1 final and only get errors. Looking forward to other responses. — vangorra, Feb 20 '15 at 22:24
Related issue for gradle-android: https://code.google.com/p/android/issues/detail?id=144664 — G. Lombard, Feb 26 '15 at 02:31
It seems that you are trying to generate coverage for a multi-project setup? I have a similar problem before, and detailed how I resolve it in my blog: http://hidroh.github.io/2014/05/15/generate-jacoco-multi-project-gradle/ — hidro, Feb 26 '15 at 09:34
UPDATE_2 works for me on a library project with gradle 1.1.3 — Grantland Chew, May 05 '15 at 22:12
So were you able to get code coverage statistics? I'm only able to get code coverage statistics for "androidTest", but I want "test" to be analyzed. — EGHDK, Jul 27 '15 at 20:09

score 20 · Accepted Answer · edited Sep 17 '21 at 06:32

20

After the hassle, I decided to create an open source Gradle plugin for that.

Root build.gradle

buildscript {
    repositories {
        mavenCentral() // optional if you have this one already
    }
    dependencies {
        classpath 'com.vanniktech:gradle-android-junit-jacoco-plugin:0.16.0'
    }
}

apply plugin: 'com.vanniktech.android.junit.jacoco'

Then simply execute

./gradlew jacocoTestReportDebug

It'll run the JUnit tests in Debug Mode and then give you the Jacoco output in XML and HTML form in the corresponding build directory.

It also supports flavors. Having 2 flavors red and blue those tasks would be created

jacocoTestReportRedDebug
jacocoTestReportBlueDebug
jacocoTestReportRedRelease
jacocoTestReportBlueRelease

edited Sep 17 '21 at 06:32

lupaulus

358
5
22

answered Oct 11 '15 at 11:36

Niklas

23,674
33
131
170

Thanks a lot for this - but when I try using your plugin as above I get the following error: java.lang.UnsupportedClassVersionError: com/vanniktech/android/junit/jacoco/Generation : Unsupported major.minor version 52.0 Any ideas? – danwilkie Oct 16 '15 at 15:14
Looks like I needed Java 8 for it to work - I've now got a report working. Thank you!! – danwilkie Oct 16 '15 at 16:04
@danwilkie yup you need Java 8 since I compiled the version with Java 8. If this helped you, feel free to upvote it. – Niklas Oct 17 '15 at 19:28
It works flawlessly, in my project I had the jcenter() in repositories and still worked (just in case somebody wonders). Great job. – markdrake Jul 06 '16 at 17:27
1

@danwilkie with the recent versions you do not need Java 8 anymore. Java 7 is enough – Niklas Aug 01 '16 at 10:14

score 8 · Answer 2 · answered Apr 13 '15 at 14:09

8

After some additional search I've stumbled upon this project I had to make some modifications so that there solution can work for my type of project, but now the test coverage reports are generated properly.

I've pushed the adopted changes to my example github repo in case someone will have a similar problem in the future.

answered Apr 13 '15 at 14:09

Serj Lotutovici

4,370
5
32
41

3

could you please explain to us what are these modifications and what you have stumbled upon please ! – Bhargav Dec 18 '15 at 19:52
I'd vote for changing the correct answer to the one provided by Niklas as it's an easy to implement solution. – markdrake Jul 06 '16 at 17:35

score 3 · Answer 3 · answered Feb 21 '15 at 17:13

Warning: This is a hack! Using your configuration above, I put together a hack to switch the android plugin between application and library depending on the build tasks chosen. This works well for me because I don't end up committing code with the application mode set.

// dynamically change the android plugin to application if we are running unit tests or test reports.
project.ext.androidPlugin = 'com.android.library'
for (String taskName : project.gradle.startParameter.taskNames) {
    if (taskName.contains('UnitTest') || taskName.contains('jacocoTestReport')) {
        project.ext.androidPlugin = 'com.android.application'
        break
    }
}

logger.lifecycle("Setting android pluging to ${project.ext.androidPlugin}")
apply plugin: project.ext.androidPlugin

...

apply plugin: 'jacoco'

configurations {
    jacocoReport
}

task jacocoTestReport(type:JacocoReport, dependsOn: "testDebug") {
    group = "Reporting"
    description = "Generate Jacoco coverage reports"

    classDirectories = fileTree(
            dir: "${buildDir}/intermediates/classes/debug",
            excludes: ['**/R.class',
                       '**/R$*.class',
                       '**/*$ViewInjector*.*',
                       '**/BuildConfig.*',
                       '**/Manifest*.*']
    )

    sourceDirectories = files("${buildDir.parent}/src/main/java")
    additionalSourceDirs = files([
            "${buildDir}/generated/source/buildConfig/debug",
            "${buildDir}/generated/source/r/debug"
    ])
    executionData = files("${buildDir}/jacoco/testDebug.exec")

    reports {
        xml.enabled = true
        html.enabled = true
    }
}

Let's hope the android tools team fixes this soon.

Unfortunately, for me it wont work, because some of my modules depend on on other modules. And this will not work when trying to run unit tests for the application module. Thx for the response though. — Serj Lotutovici, Feb 23 '15 at 08:27

score 3 · Answer 4 · edited Oct 29 '15 at 01:17

I setup my unit tests for gradle 1.2 using this blog post. Then I pieced together information I found here and elsewhere to add code coverage to independent modules instead of the whole project. In my library module build.gradle file, I added the following:

apply plugin: 'jacoco'

def jacocoExcludes = [
        'com/mylibrary/excludedpackage/**'
]

android {
    ...
}

android.libraryVariants.all { variant ->
    task("test${variant.name.capitalize()}WithCoverage", type: JacocoReport, dependsOn: "test${variant.name.capitalize()}") {
        group = 'verification'
        description = "Run unit test for the ${variant.name} build with Jacoco code coverage reports."

        classDirectories = fileTree(
                dir: variant.javaCompile.destinationDir,
                excludes: rootProject.ext.jacocoExcludes.plus(jacocoExcludes)
        )
        sourceDirectories = files(variant.javaCompile.source)
        executionData = files("${buildDir}/jacoco/test${variant.name.capitalize()}.exec")

        reports {
            xml.enabled true
            xml.destination "${buildDir}/reports/jacoco/${variant.name}/${variant.name}.xml"
            html.destination "${buildDir}/reports/jacoco/${variant.name}/html"
        }
    }
}

And in my project build.gradle file, I added common excludes:

ext.jacocoExcludes = [
    'android/**',
    '**/*$$*',
    '**/R.class',
    '**/R$*.class',
    '**/BuildConfig.*',
    '**/Manifest*.*',
    '**/*Service.*'
]

Also, it looks like code coverage for unit tests may be coming built in in the future Issue 144664

score 2 · Answer 5 · answered Mar 18 '15 at 13:55

I was finally able to see my code coverage of JUnit tests with Android Studio 1.1.

jacoco.gradle

apply plugin: 'jacoco'

jacoco {
    toolVersion "0.7.1.201405082137"
}

def coverageSourceDirs = [
        "$projectDir/src/main/java",
]

task jacocoTestReport(type: JacocoReport, dependsOn: "testDebug") {
    group = "Reporting"
    description = "Generate Jacoco coverage reports after running tests."
    reports {
        xml.enabled = true
        html.enabled = true
    }
    classDirectories = fileTree(
            dir: './build/intermediates/classes/debug',
            excludes: ['**/R*.class',
                       '**/*$InjectAdapter.class',
                       '**/*$ModuleAdapter.class',
                       '**/*$ViewInjector*.class'
            ]
    )
    sourceDirectories = files(coverageSourceDirs)
    executionData = files("$buildDir/jacoco/testDebug.exec")
    // Bit hacky but fixes https://code.google.com/p/android/issues/detail?id=69174.
    // We iterate through the compiled .class tree and rename $$ to $.
    doFirst {
        new File("$buildDir/intermediates/classes/").eachFileRecurse { file ->
            if (file.name.contains('$$')) {
                file.renameTo(file.path.replace('$$', '$'))
            }
        }
    }
}

and then within the build.gradle file of the module (I put it between android and dependencies):

apply from: '../jacoco.gradle'

Also in the defaultConfig block of android. I've added this (don't know if it is necessary, but I've got this from this blog):

android {
    defaultConfig {
        testHandleProfiling true
        testFunctionalTest true
    }
}

Enjoy.

This generated a report for me... but it just shows how many tests I have and if they failed. Any way to get Code Coverage results? — EGHDK, Jul 27 '15 at 20:08

Artur Stepniewski · Answer 6 · 2015-11-30T19:59:42.103

2

You can try to use this Gradle plugin: https://github.com/arturdm/jacoco-android-gradle-plugin

Basically, all you need to do is apply it like this:

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath 'com.dicedmelon.gradle:jacoco-android:0.1.1'
  }
}

apply plugin: 'com.android.library' // or 'com.android.application'
apply plugin: 'jacoco-android'

As a result you should get a JacocoReport task for each variant. Run the command below to generate code coverage reports for all of them.

$ ./gradlew jacocoTestReport

edited Nov 30 '15 at 19:59

answered Sep 14 '15 at 19:16

Artur Stepniewski

1,401
2
11
10

Like the idea. Does it aggregate the reports in one? – Serj Lotutovici Sep 14 '15 at 22:25
You get separate reports for each variant. That's the same behavior as with instrumentation tests. The output directory for those variants is `build/reports/jacoco/jacoco${testTaskNameOfVariant}Report`. – Artur Stepniewski Sep 15 '15 at 16:24
1

@behelit, you can see this article http://allegro.tech/2016/03/integrating-android-project-with-codecov.html for some more information on those reports and if you still get stuck at some point it would be helpful if you provided more details. – Artur Stepniewski Mar 18 '16 at 22:33

score 1 · Answer 7 · answered Mar 10 '15 at 22:25

1

I resolve issues with JaCoCo and make it work with latest gradle android plugin 1.1.3

Project with latest gradle scripts: https://github.com/OleksandrKucherenko/meter

References:

How to attach own implementation instead of Mocks in Android Studio Unit Tests? https://plus.google.com/117981280628062796190/posts/8jWV22mnqUB

Small hint for everyone who try to use JaCoCo coverage in android builds... unexpected finding!!! https://plus.google.com/117981280628062796190/posts/RreU44qmeuP

JaCoCo XML/HTML Report for Unit Tests https://plus.google.com/u/0/+OleksandrKucherenko/posts/6vNWkkLed3b

answered Mar 10 '15 at 22:25

Oleksandr Kucherenko

1,921
1
17
21

btw JaCoCo, PMD, Findbugs, Checkstyle are also working in project. – Oleksandr Kucherenko Mar 10 '15 at 22:26
Does this *require* 1.1.3? Or just that you are using 1.1.3? – TJR Mar 18 '15 at 18:45

score 1 · Answer 8 · answered Mar 19 '15 at 19:49

I was facing exactly the same problem like you. Today I did completely removed android studio, android sdk, gradle. Then reinstall everything. After that, I just added inside the app build.gradle.

debug { testCoverageEnabled true } Then I run ./gradlew connectedChec. Everything is working perfectly. Android studio default Jacoco working fine for me. I think it is also possible to create a jacocoTestReport Task and then create code coverage.I don't know why gradle and android studio was not working previously.

score 0 · Answer 9 · answered Feb 26 '15 at 09:18

Please create an example and I can take a look. I guess it's some missing path configuration.

include all coverage files (*.exec)
add all your source paths (module/src/main/java)
add all class paths (module/build/intermediates/classes/debug)

here two examples how it could be look

Jacoco and Unit Tests Code Coverage with android-gradle-plugin >= 1.1

9 Answers9

Linked