13

I've tried to use kcov to get code coverage for a Rust library. I've followed this tutorial to build and use kcov. The coverage seems to work, however I'm facing a strange high coverage. Some files in the project gets a 100% coverage, even if they are actually not covered at all!

This is a minimal project reproducing the problem:

Cargo.toml

[package]
name = "mypackage"
version = "0.1.0"
authors = ["mbrt"]

src/lib.rs

pub mod subm;

pub fn coverage1(i : bool) -> bool {
    if i {
        true
    }
    else {
        false
    }
}

#[cfg(test)]
mod test {
    use super::coverage1;

    #[test]
    fn test_coverage1() {
        assert!(coverage1(true));
    }
}

src/subm.rs

pub fn coverage2(i : bool) -> bool {
    if i {
        true
    }
    else {
        false
    }
}

#[cfg(test)]
mod test {
    #[test]
    fn test_coverage2() {
    }
}

There are two identical functions, one in the root of the crate, and another in a submodule. The only difference is that the first test stimulates one function, and the other does nothing at all. In this case I'd expect a coverage not greater than 50%.

However kcov reports this:

coverage summary

The coverage for lib.rs is correct:

coverage1

But the coverage for subm.rs is wrong! Note that the function is public, so it cannot be optimized out from the library:

coverage2

Here we can verify that kcov is working, because it is able to compute code coverage for one file, but it is not able to see that the second file is not covered at all.

What is the problem here? Maybe test binaries strip down unused functions and kcov cannot see them?

mbrt
  • 1,958
  • 1
  • 17
  • 33
  • 4
    *the function is public, so it cannot be optimized out from the library* — it absolutely can be. Everything is statically compiled, so the compiler knows which functions are used (and how!) when it creates the binary. This is a huge benefit due to monomorphization. – Shepmaster Sep 11 '15 at 14:57
  • @Shepmaster Good point, I didn't think about it. – mbrt Sep 11 '15 at 14:58

2 Answers2

14

There is a workaround: the RUSTFLAGS='-C link-dead-code' environment variable. Use it while building and the Rust compiler will link dead code as well:

RUSTFLAGS='-C link-dead-code' cargo test
mbrt
  • 1,958
  • 1
  • 17
  • 33
6

You're correct: totally unused functions are stripped at the moment, so coverage tools like kcov are only good for branch coverage within used functions (at least, the summary functionality of such tools). There is some discussion about making this not happen by default for test/debug builds.

huon
  • 94,605
  • 21
  • 231
  • 225
  • I suppose that at the very least the functions should remain when the "coverage" switch is turned on. – Matthieu M. Sep 11 '15 at 13:39
  • @MatthieuM. The problem is that this is not the case, as far as I can see. – mbrt Sep 11 '15 at 14:49
  • 1
    @brt: Indeed, which is fairly amusing. – Matthieu M. Sep 11 '15 at 15:11
  • @MatthieuM. there's no "coverage" switch at the moment (kcov just piggybacks on the standard DWARF debug info). – huon Sep 12 '15 at 00:45
  • @huon-dbaupp: to be honest, I would say this is a bug in `kcov` (though it is likely due to it being language agnostic); at the very least it should report the percentage of ignored lines (and which they are). – Matthieu M. Sep 12 '15 at 12:35
  • I don't see how kcov can know about functions that don't appear in the binary. (It could just assume that all lines that aren't mentioned in debuginfo are code lines that weren't hit... but there's a lot of comments/whitespace/`}`s in most programs, to the point of making that metric useless.) – huon Sep 12 '15 at 12:39
  • @huon-dbaupp Is there any workaround allowing to build without stripping down unused functions? – mbrt Sep 12 '15 at 21:07