Native Code coverage with android soong build system

Question

I'm trying to generate code coverage report for my native components with AOSP source code using soong build system.

I have extended aosp vhal but unit test cases are same as in below link. http://androidxref.com/8.1.0_r33/xref/hardware/interfaces/automotive/vehicle/2.0/default/tests/

Tried adding below to cc_test, cc_binary in Android.bp

native_coverage : true,

    cflags: [
         "-g",
         "-O0",
         "-fprofile-arcs",
         "-ftest-coverage",
    ],

     ldflags : [
        "-fprofile-arcs",
        "-ftest-coverage",
    ],

Native binary unit-tests-coverage is generated in out/target/product but I can't find gcno intermediates for this.

Running below command gives me *.gcda files for each test files.

adb shell \
    GCOV_PREFIX=/data/local/tmp \
    GCOV_PREFIX_STRIP=`echo $ANDROID_BUILD_TOP | grep -o / | wc -l` \
    /data/local/tmp/unit-tests-coverage

I have tried below links but not sure how to proceed :(

http://logan.tw/posts/2015/04/28/check-code-coverage-with-clang-and-lcov/ https://android.googlesource.com/platform/bionic.git/+/master-soong https://android.googlesource.com/platform/build/soong/+/581341d%5E%21/ https://android.googlesource.com/platform/external/e2fsprogs/+/fedfb27%5E%21/ https://android.googlesource.com/platform/development/+/master/scripts/acov#23 http://androidxref.com/9.0.0_r3/xref/bionic/README.md#293

I'm not sure if google's vts framework can be used here to generate native code coverage. https://codelabs.developers.google.com/codelabs/android-vts-8/#6

"gcnodir" is generated but not sure how to make use of it. /coverage/data/nativetest64/vehicle-unit-tests-coverage/unit-tests-coverage.gcnodir

Answer 1

Posting answer to my question for other users on SO.

Install coverage tool :

1. sudo apt-get install lcov (This should install lcov-1.12)
2. sudo apt-get install gcc-4.6 (Clang generates .gcno approximately equal to gcc 4.2 that aren't compatible with gcov-4.8. Installing gcc-4.6 to get gcov-4.6 and invoking lcov with '--gcov-tool /usr/bin/gcov-4.6')
3. Download LLVM 3.8 for llvm-cov to work : http://releases.llvm.org/download.html

All native unit test cases i.e instrumented binary needs to be executed on target. To build and emit clang's instrumentation based profiling. Example: http://androidxref.com/9.0.0_r3/xref/hardware/interfaces/automotive/vehicle/2.0/default/Android.bp#82 (Renamed to vehicle-manager-unit-test for shorter name)

• export NATIVE_COVERAGE=true
• Add native_coverage: true to test module in Android.bp
• Go to: module-name/test
• Use mm or make command to build native binary
• Ex: For hardware/interfaces/automotive/vehicle/2.0/default/tests/ : mma or make vehicle-manager-unit-test -j32
• Copy coverage enabled instrumented binary to target

• adb push out/target/product/product_name/data/nativetest64/vendor/vehicle-manager-unit-test /data/nativetest64/vehicle-manager-unit-test adb shell chmod +x /data/nativetest64/vehicle-manager-unit-test

• Run test cases and generate .gcda files

  adb shell \ GCOV_PREFIX=/data/local/tmp \ GCOV_PREFIX_STRIP=echo $ANDROID_BUILD_TOP | grep -o / | wc -l \ /data/nativetest64/vehicle-manager-unit-test

• adb shell find -iname *.gcda

• adb pull /data/local/tmp/proc/self/cwd/out/soong/.intermediates/hardware/interfaces/automotive/vehicle/2.0/default/vehicle-manager-unit-test/android_x86_64_silvermont_vendor_cov/obj/hardware/interfaces/automotive/vehicle/2.0/default/tests/ .(Destination folder)

• Extract GCNO files from GCNODIR (archive file generated at out/overage/data/nativetest64/vendor/vehicle-manager-unit-test ) to same folder with GCDA files

• llvm-cov gcov -f -b *.gcda (https://llvm.org/docs/CommandGuide/llvm-cov.html )

• lcov --directory . --base-directory . --gcov-tool /usr/bin/gcov-4.6 --capture -o cov.info (http://ltp.sourceforge.net/coverage/lcov.php)

• genhtml cov.info -o output

Here's the script which wraps all these commands: https://gist.github.com/pankajgangwar/f070b8b54e83543f8e3638dcd2cae1b8

Answer 2

here it is explaned how to generate coverage reports, which do require GTest:

these flags enable the generation of test coverage: -fprofile-arcs -ftest-coverage

then one has to use gcov: gcov main_test.cpp

which's output then can be passed on to lcov (for reference):

$ lcov --coverage --directory . --output-file main_coverage.info

from which one can generate an lcov coverage report in HTML format:

$ genhtml main_coverage.info --output-directory out

---

these .gcda files in .gcnodir are gcov data files. gcov also has an output option --json-format, which might come handy when wanting to consume the coverage data with a web-service.

---

one of the examples from the links you've provided can be used to generate it for a whole project:

Collect the code coverage results:

$ lcov --directory . \
   --base-directory . \
   --gcov-tool gcov.sh \
   --capture -o cov.info

Generate HTML files:

$ genhtml cov.info -o output

where the only difference is, that the wrapper script would need to be adjusted to call gcov. probably one could even omit the wrapper passed with option --gcov-tool, since it should be directly called.

---

since one can only prepare the coverage report by adding the compiler flags, the gcov and lcov commands should be setup as post-build script, so that they would automatically generate the report.