sbt assembly with integration test

Question

Hi i have an SBT build file that handles Integration test within our organization, the test themselves work and i can run separately unit and it test. However when packaging the app with sbt-assembly i am not able to run both test sequentially (unit ++ it) because it goes out of memory when compiling, here is the sbt.Project with its settings:

  import BuildSettings._
  import Dependencies._
  import Resolvers._

  val name = "api"
  lazy val api = Project(
    name, file("."),
    settings = buildSettings
      ++ Seq(resolvers := repositories, libraryDependencies ++= dependencies)
      ++ SbtStartScript.startScriptForClassesSettings
      ++ Revolver.settings
      ++ assemblySettings
      ++ Seq(jarName := name + "-" + currentGitBranch + ".jar")
  ).configs(IntegrationTest)
    .settings(Defaults.itSettings: _*)
    .settings(parallelExecution in IntegrationTest := false)
    .settings(parallelExecution in Test := true)
    .settings(scalaSource in IntegrationTest := baseDirectory.value / "src/test/scala")
    .settings(resourceDirectory in IntegrationTest := baseDirectory.value / "src/test/resources")
    .settings(
  testOptions in IntegrationTest := Seq(Tests.Filter(itFilter)),
  testOptions in Test := Seq(Tests.Filter(unitFilter))
    )
    .settings(parallelExecution in Compile := true)
    .settings(fork in Compile := true)
    .settings(sources in (Compile, doc) := Seq.empty)
    .settings(test in assembly := Seq(
      (test in Test).value ,
      (test in IntegrationTest).value
    ))

  def currentGitBranch = {
    "git rev-parse --abbrev-ref HEAD".lines_!.mkString.replaceAll("/", "-").replaceAll("heads-", "")
  }

  def itFilter(name: String): Boolean = name endsWith "IT"

  def unitFilter(name: String): Boolean = !itFilter(name)

as you can see i used (test in assembly := { ... }) to specify what test to run during the task assembly::test, however if i run sbt assembly it fails with

java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:GC overhead limit exceeded

I tried using only one of the two (unit/it) test and it manages to compile but i do need to have them both, any suggestion? Thanks in advance.

Edit: i tried running sbt with SBT_OPTS="-Xmx1536M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled" and i had a different message in the error, java.lang.OutOfMemoryError: Java heap space

score 8 · Accepted Answer · answered Dec 23 '16 at 18:16

8

What you're doing now is running test and integration test in parallel during the assembly. SBT tries to parallelize all the things by running independent operations in parallel, and definition like this:

test in assembly := Seq(
  (test in Test).value ,
  (test in IntegrationTest).value
)

allows it to do so, because those 2 tasks (test in Test and test in IntegrationTest) are independent from each other.

It might work if you give it more memory with sbt -mem 4096 or even more, but you can also run them sequentially using Def.sequential approach, as described here. Your code would look like this:

test in assembly := Def.sequential(
  test in Test,
  test in IntegrationTest
).value

answered Dec 23 '16 at 18:16

Haspemulator

11,050
9
49
76

Thanks, executing them sequentially made the trick and it doesn't take all the memory to do that, luckily i was already on SBT 0.13.8 as it seems they introduced "Def.sequential" in that release. – user145634 Dec 29 '16 at 09:05
Glad it helped, and a general comment: SBT is pretty much backwards compatible, so there's really no reason not to update to the most recent version, 0.13.13 currently. – Haspemulator Dec 29 '16 at 17:42

sbt assembly with integration test

1 Answers1