37

I am generating jacoco report by using jacoco:report tag. I am getting errors like :

[jacoco:report] Classes in bundle 'Code Coverage Report' do no match with execution data. For report generation the same class files must be used as at runtime.
[jacoco:report] Execution data for class xxxxx does not match.
[jacoco:report] Execution data for class yyyyy does not match.

The ant report target looks like :

<target name="report">
                <jacoco:report>
                        <executiondata>
                                <file file="${jacocoexec.dir}/${jacocoexec.filename}"/>
                        </executiondata>
                        <!-- the class files and optional source files ... -->
                        <structure name="Code Coverage Report">
                                <classfiles>
                                        <fileset file="./jar/abc.jar"/>
                                </classfiles>
                                <sourcefiles>
                                      <fileset dir="./code/src"/>
                                </sourcefiles>
                        </structure>
                        <!-- to produce reports in different formats. -->
                        <html destdir="${jacoco.report.dir}"/>
                </jacoco:report>
        </target>

The abc.jar so generated is by using ./code/src only. Then why is it giving such errors. Any idea?

Nishant Lakhara
  • 2,295
  • 4
  • 23
  • 46
  • Add how executiondata (`${jacocoexec.filename}`) is generated? There should be separate target which executes `jacoco:coverage` task. Put this target to question as well. – Tagir Valeev Jul 30 '15 at 11:37
  • by adding -javaagent parameter while running test case: -javaagent:JaCoCoProject/lib/jacocoagent.jar=destfile=JaCoCoProject/jacoco.exec,output=file – Nishant Lakhara Jul 30 '15 at 12:06
  • 1
    please share the full build file. All that error is saying is you instrumented a jar, ran tests- Those classes are not from "./jar/abc.jar"- It is not enough to match the classes by name. they must match the "classID" seen by jacoco- which means even if you recompile, the classes are different. – Jayan Aug 08 '15 at 16:30
  • @Jayan : Does recompiling on same code may change the content in jar ? – Nishant Lakhara Aug 10 '15 at 09:19
  • Yes- from jacoco's point of view. It computes has hash from binary- this is different each time you compile- probably due to timestamps. – Jayan Aug 10 '15 at 09:23
  • @Nishu: you are offering a bounty- The OP however did not add enough details to the qustion- do you have access to smaple build.xml used by OP(or whatever the question leads to?) – Jayan Aug 10 '15 at 09:26
  • @Jayan: Nope... actually i am getting such errors while generating code coverage in my project and that's why set a bounty to this one. Your comment looked a bit convincing and that's why asked you – Nishant Lakhara Aug 10 '15 at 09:31
  • @Nishu : So you can edit question with your additional details :). Please see my answers - which is a shameless copy from JaCoCo Site. – Jayan Aug 10 '15 at 10:02

5 Answers5

41

You are getting the error related to classID. This is a concept described in detail at JaCoCo docs-site. http://www.eclemma.org/jacoco/trunk/doc/classids.html. This is a key step for supporting multiple versions of class (an appserver for example) in same JVM.

Copying part some part of it here for visibility.

What are class ids and how are they created?

Class ids are 64-bit integer values, for example 0x638e104737889183 in hex notation. Their calculation is considered an implementation detail of JaCoCo. Currently ids are created with a CRC64 checksum of the raw class file.

What can cause different class ids?

Class ids are identical for the exact same class file only (byte-by-byte). There is a couple of reasons why you might get different class files. First compiling Java source files will result in different class files if you use a different tool chain:

  • Different compiler vendor (e.g. Eclipse vs. Oracle JDK)

  • Different compiler versions

  • Different compiler settings (e.g. debug vs. non-debug)

Also post-processing class files (obfuscation, AspectJ, etc.) will typically change the class files. JaCoCo will work well if you simply use the same class files for runtime as well as for analysis. So the tool chain to create these class files does not matter.

Even if the class files on the file system are the same there is possible that classes seen by the JaCoCo runtime agent are different anyways. This typically happens when another Java agent is configured before the JaCoCo agent or special class loaders pre-process the class files. Typical candidates are:

  • Mocking frameworks
  • Application servers
  • Persistence frameworks

The same page covers possible solutions.

What workarounds exist to deal with runtime-modified classes?

If classes get modified at runtime in your setup there are some workarounds to make JaCoCo work anyways:

  • If you use another Java agent make sure the JaCoCo agent is specified at first in the command line. This way the JaCoCo agent should see the original class files.
  • Specify the classdumpdir option of the JaCoCo agent and use the dumped classes at report generation. Note that only loaded classes will be dumped, i.e. classes not executed at all will not show-up in your report as not covered.
  • Use offline instrumentation before you run your tests. This way classes get instrumented by JaCoCo before any runtime modification can take place. Note that in this case the report has to be generated with the original classes, not with instrumented ones.

Edited on 22-02-2017

How to use Offline Instrumentation: Use below task provided by Daniel Atallah.

//Additional SourceSets can be added to the jacocoOfflineSourceSets as needed by 
project.ext.jacocoOfflineSourceSets = [ 'main' ]
task doJacocoOfflineInstrumentation(dependsOn: [ classes, project.configurations.jacocoAnt ]) {
    inputs.files classes.outputs.files
    File outputDir = new File(project.buildDir, 'instrumentedClasses')
    outputs.dir outputDir
    doFirst {
        project.delete(outputDir)
        ant.taskdef(
            resource: 'org/jacoco/ant/antlib.xml',
            classpath: project.configurations.jacocoAnt.asPath,
            uri: 'jacoco'
        )
        def instrumented = false
        jacocoOfflineSourceSets.each { sourceSetName ->
            if (file(sourceSets[sourceSetName].output.classesDir).exists()) {
                def instrumentedClassedDir = "${outputDir}/${sourceSetName}"
                ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
                    fileset(dir: sourceSets[sourceSetName].output.classesDir, includes: '**/*.class')
                }
                //Replace the classes dir in the test classpath with the instrumented one
                sourceSets.test.runtimeClasspath -= files(sourceSets[sourceSetName].output.classesDir)
                sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
                instrumented = true
            }
        }
        if (instrumented) {
            //Disable class verification based on https://github.com/jayway/powermock/issues/375
            test.jvmArgs += '-noverify'
        }
    }
}
test.dependsOn doJacocoOfflineInstrumentation

Now generate report using "gradlew test jacocoTestReport" command.

Sagar Trehan
  • 2,401
  • 2
  • 24
  • 32
Jayan
  • 18,003
  • 15
  • 89
  • 143
  • 3
    Thanks for adding this answer. its quite helpful in understanding the jacoco framework and associated concepts. – Nishant Lakhara Aug 10 '15 at 10:30
  • 3
    @Jayan: Your answer is correct but very difficult to understand what exactly I have to do to fix the issue. If you can provide some code snippet then it will be very easier for me. You have written the answer considering that the person who will read this have good understanding of how Jacoco works. – Sagar Trehan Feb 22 '17 at 07:40
  • 1
    @Jayan: Edited your answer and added a task that I used for offline instrumentation. Please review. – Sagar Trehan Feb 22 '17 at 07:47
  • I get error ` build file 'C:\Projects\projname\subdir\build.gradle': 45: expecting ''', found '\r' @ line 45, column 51. : 'org/jacoco/ant/antlib.xml',` Any idea what is this? – Carmageddon Jan 25 '18 at 17:43
  • Could not get unknown property 'classesDir' for main classes of type org.gradle.api.internal.tasks.DefaultSourceSetOutput. – Freedom Dec 28 '20 at 10:03
6

JaCoCo needs the exact same class files for report generation that used at execution time. Due to different compilers and/or other tools modifying the classes, the classes might be different.

kkmonlee
  • 379
  • 2
  • 16
4

I have noticed that this happens if a class you want to report code coverage for has its static initialization suppressed by Mockito with PowerMockito in your JUnit test. For example, if your test class looks like:

@SuppressStaticInitializationFor(
        {"com.yourpkg.A",
        "com.yourpkg.B"})
public class Test {
     @Test
     public void Test() { }
}

The error when you test will be something like:

Classes in bundle 'yourProject' do not match with execution data. 
For report generation the same class files must be used as at runtime. 
Execution data for class com/yourpkg/A does not match. 
Execution data for class com/yourpkg/B does not match.
Calicoder
  • 1,322
  • 1
  • 19
  • 37
  • I have the same problem with PowerMock. I'm not using that annotation, but with `@RunWith(PowerMockRunner::class)` and `@PrepareForTest(TestedClass::class)` (kotlin) – HendraWD Apr 20 '21 at 10:28
  • @HendraWD did you ever figure out a fix? I'm encountering the same issue – RCB Oct 19 '21 at 17:02
0

Various answers with deep insights, but still I will share what worked for me after 2 days of fiddling with the setup.

  • I am using Azure Devops for git.

So as per the code setup guideline, it says that you need to run the analysis task after your build task. And this was the step I was completely missing.

Here is how my task sequence looked before

 - task: Gradle@2
    inputs:
      gradleWrapperFile: 'gradlew'
      tasks: '--build-cache jacocoTestReport SonarQube'
      options: '-PversionName=$(Build.BuildNumber) -PversionCode=$(Build.BuildId) -Porg.gradle.parallel=true'
      publishJUnitResults: true
      testResultsFiles: '**/TEST-*.xml'
      javaHomeOption: 'JDKVersion'
      jdkVersionOption: '1.11'
      gradleOptions: '-Xmx4096m'
      sonarQubeRunAnalysis: false
    continueOnError: true
    displayName: "Run Quality Scan and upload"
    env:
      SONAR_LOGIN: $(sonar-login)
      BRANCH_NAME: $(Build.SourceBranchName)
      
  - task: Gradle@2
    condition: in(variables['Build.SourceBranchName'], 'develop')
    inputs:
      gradleWrapperFile: 'gradlew'
      tasks: '--build-cache build publishAllPublicationsToMyDigitalRepository'
      options: '-PversionName=$(Build.BuildNumber) -PversionCode=$(Build.BuildId) -PsecureSign'
      publishJUnitResults: true
      testResultsFiles: '**/TEST-*.xml'
      javaHomeOption: 'JDKVersion'
      jdkVersionOption: '1.11'
      gradleOptions: '-Xmx4096m'
      sonarQubeRunAnalysis: false
    env:
      SDK_NAME: 'iphoneos'
    continueOnError: false
    displayName: "Build & publish library"

And here is how it looked after the repositioning

  - task: Gradle@2
    condition: in(variables['Build.SourceBranchName'], 'develop')
    inputs:
      gradleWrapperFile: 'gradlew'
      tasks: '--build-cache build publishAllPublicationsToMyDigitalRepository'
      options: '-PversionName=$(Build.BuildNumber) -PversionCode=$(Build.BuildId) -PsecureSign'
      publishJUnitResults: true
      testResultsFiles: '**/TEST-*.xml'
      javaHomeOption: 'JDKVersion'
      jdkVersionOption: '1.11'
      gradleOptions: '-Xmx4096m'
      sonarQubeRunAnalysis: false
    env:
      SDK_NAME: 'iphoneos'
    continueOnError: false
    displayName: "Build & publish library"

  - task: Gradle@2
    inputs:
      gradleWrapperFile: 'gradlew'
      tasks: '--build-cache jacocoTestReport SonarQube'
      options: '-PversionName=$(Build.BuildNumber) -PversionCode=$(Build.BuildId) -Porg.gradle.parallel=true'
      publishJUnitResults: true
      testResultsFiles: '**/TEST-*.xml'
      javaHomeOption: 'JDKVersion'
      jdkVersionOption: '1.11'
      gradleOptions: '-Xmx4096m'
      sonarQubeRunAnalysis: false
    continueOnError: true
    displayName: "Run Quality Scan and upload"
    env:
      SONAR_LOGIN: $(sonar-login)
      BRANCH_NAME: $(Build.SourceBranchName)

So in essence I learned a few things, out of context here but most important one was.

  • right order of tasks
  • and for Sonar to correctly find those files was, to have that build available.
sud007
  • 5,824
  • 4
  • 56
  • 63
0

After adding offline instrumentation, coverage was proper but I started getting class is already instrumented message. Adding this in build.gradle helps.

test {
    jacoco {
        excludes = ['{package_path}']     //package_path like '*com/**'
    }
}
Josh
  • 1
  • 2