15

Am running the integration test using following sbt command

sbt clean coverage it:test coverageReport

This command runs integration tests, instruments it and generates report as well.

Build.sbt has following:

coverageMinimum in IntegrationTest := 21.0
coverageFailOnMinimum in IntegrationTest := true

Output looks like:

[info] Statement coverage.: 20.16%
[info] Branch coverage....: 12.00%
[info] Coverage reports completed
[info] All done. Coverage was [20.16%]

Output result has 20.16% code coverage but the limits in build.sbt are not enforcing the limit.

If I change build.sbt to following it works:

coverageMinimum := 21.0
coverageFailOnMinimum := true

Wanted to know what am I missing for specifying limits specifically for Integration tests

Version Information:

sbt : 0.13.17

sbt-scoverage : 1.5.1

aditya parikh
  • 595
  • 4
  • 11
  • 30
  • 1
    It works with the appropriate setting `coverageFailOnMinimum`, which happens not to be set by default, so what? – cchantep Sep 11 '18 at 07:15
  • What do you mean by appropriate setting ? Like the same two settings defined within Integration scope doesn’t seem to be recognized. What am I missing ? – aditya parikh Sep 11 '18 at 13:24

1 Answers1

5

The following two workarounds seem to work on my machine (sbt-scoverage 1.5.1, sbt 1.1.1, scala 2.12.5)

Workaround 1 - Use inConfig to scope to a configuration:

inConfig(IntegrationTest)(ScoverageSbtPlugin.projectSettings),
inConfig(IntegrationTest)(Seq(coverageMinimum := 21, coverageFailOnMinimum := true))

Now executing sbt clean coverage it:test it:coverageReport throws Coverage minimum was not reached.

Workaround 2 - Modify coverageMinimum setting within a custom command:

def itTestWithMinCoverage = Command.command("itTestWithMinCoverage") { state =>
  val extracted = Project extract state
  val stateWithCoverage = extracted.append(Seq(coverageEnabled := true, coverageMinimum := 21.0, coverageFailOnMinimum := true), state)
  val (s1, _) = Project.extract(stateWithCoverage).runTask(test in IntegrationTest, stateWithCoverage)
  val (s2, _) = Project.extract(s1).runTask(coverageReport in IntegrationTest, s1)
  s2
}

commands ++= Seq(itTestWithMinCoverage)

Now executing sbt itTestWithMinCoverage throws Coverage minimum was not reached. Note after executing itTestWithMinCoverage the state is discarded so coverageMinimum should be back to default value, and thus not affect unit tests.

It seems the issue is (besides my lack of understanding how scopes exactly work) checkCoverage picks up default value of coverageMinimum even after setting coverageMinimum in IntegrationTest.

Mario Galic
  • 47,285
  • 6
  • 56
  • 98
  • I tried the inconfig, 1st approach, didn't work for me. have an older version of scala then yours. i defined for test and integration test but it doesn't seem to read it – aditya parikh Sep 19 '18 at 19:55