Sequencing and overriding tasks in SBT

Question

Quick summary: I'm trying to wait in the top-level project for all SBT submodules to build, then remove their target directories. Top-level application aggregates all submodules, they won't be deployed separately but only as a bundle with classpath dependencies, while duplicated libraries in submodules blow up the size of the whole package and the slug goes over Heroku limit.

Technically, I'm trying to actually use this - I'm trying to add a 'cleanup' task that would run after stage. The solution from link above doesn't seem to work for me (Play 2.4, SBT 0.13.5), the error says it better than I can:

build.sbt:50: error: reference to stage is ambiguous;
it is imported twice in the same scope by
import _root_.com.typesafe.sbt.packager.universal.UniversalPlugin.autoImport._
and import $52e59eb09172b3222f9e._
stage := {

Assuming I have my cleanup task:

val stageCleanupTask = taskKey[Unit]("Clean after stage task")

stageCleanupTask := {
  val log = streams.value.log
  if (sys.env.getOrElse("POST_STAGE_CLEAN", "false").equals("true")) {
    log.info("Cleaning submodules' target directories")
    sbt.IO.delete(baseDirectory.value / "modules" / "a" / "target")
    sbt.IO.delete(baseDirectory.value / "modules" / "b" / "target")
    sbt.IO.delete(baseDirectory.value / "modules" / "c" / "target")
  }
}

and I'm tring to override stage:

stage := {
  val f = (stage in Universal).value
  stageCleanupTask.value
  f
}

It seems to be simply wrong, since both tasks run concurrently. SBT doesn't really make it easy either, I didn't find much in the official documentation, so I was just playing around:

stage.flatMap expects a function that returns a Task[U], but stageCleanupTask is a TaskKey[T], and .value doesn't work outside of very specific areas, so composition via something similar to stage <<= stage.flatMap(f => stageCleanupTask.map(_ => f)) seems to be out of the question.
dependsOn could only work as stage <<= stage dependsOn stageCleanupTask, which is the exact opposite of the dependency chain I want. stageCleanupTask should depend on stage, but the types won't match (Task[Unit] vs Task[File])
I was trying to experiment with composition inside overriden stage, as in:
```
stage := {
    (stage in Universal).map(f => /*follow up*/ f).value
}
```
but that usually just slaps me in the face with illegal dynamic dependency or illegal dynamic reference

What would be the preferred way of sequencing SBT tasks?

I don't know the answer to your question, but as far as reducing the slug size of your Heroku app goes, you could fork the buildpack and modify this section to include your `target` sub-dirs: https://github.com/heroku/heroku-buildpack-scala/blob/master/bin/compile#L149-L171 — codefinger, Jan 19 '16 at 15:06
I would also accept a PR for adding an SBT_POST_TASKS here: https://github.com/heroku/heroku-buildpack-scala/blob/master/bin/compile#L109 — codefinger, Jan 19 '16 at 15:07
@codefinger Thanks, I'll definitely look into SBT_POST_TASKS - no promises though, bash (especially robust scripts) is not my forte :) — Patryk Ćwiek, Jan 19 '16 at 18:15
I made the change. https://github.com/heroku/heroku-buildpack-scala/commit/fdaa1159b8f75909e55566b12a222afef486cf05 If you can open a ticket at http://help.heroku.com I can tell you how to use this branch of the buildpack and provide more info. I'd rather get the discussion out of this thread. — codefinger, Jan 19 '16 at 21:43
@codefinger SBT only solutions should be completely possible though ;) It was called `Simple Build Tool` for a reason — Martijn, Jul 05 '16 at 23:25

score 7 · Answer 1 · answered Jul 04 '16 at 12:08

7

See http://www.scala-sbt.org/0.13/docs/Howto-Sequencing.html for how to sequence tasks.

So something like:

stage in Universal := Def.sequential(
  stage in Universal,
  stageCleanupTask
)

answered Jul 04 '16 at 12:08

Dale Wijnand

6,054
5
28
55

1

I up-voted this but after reviewing the actual code, I have second thoughts. This doesn't do what the OP requested. This is the actual signature of the method: `def sequential[A0, B](task0: Initialize[Task[A0]], last: Initialize[Task[B]]): Initialize[Task[B]]` which means it still returns the type of the second task you pass it, in this case `Unit`. This is incorrect since `stage` requires `sbt.File`. – Martijn Jul 05 '16 at 23:05

Martijn · Answer 2 · 2016-07-05T23:20:32.820

5

Let me start by saying that baseDirectory.value / "modules" / "a" / "target" is not the kind of path definition you want to use since it's much better to make use of the settings SBT provides you. What I recommend is using (target in moduleName).value.

As for your main question, I recommend you do this:

val stageCleanupTask = taskKey[sbt.File]("Clean after stage task")

lazy val root = project.in(file("."))
  ...
  .settings(
    stageCleanupTask := {
      val a = (stage in Universal).value
      val log = streams.value.log
      if (sys.env.getOrElse("POST_STAGE_CLEAN", "false").equals("true")) {
        log.info("Cleaning submodules' target directories")
        sbt.IO.delete((target in a).value)
        sbt.IO.delete((target in b).value)
        sbt.IO.delete((target in c).value)
      }
      a
    },
    stage <<= stageCleanupTask

I just tested in one of my own projects and it worked flawlessly.

Edit 1

My battery is dying so I can't look into this further but it might be the thing you're looking for.

edited Jul 05 '16 at 23:20

answered Jul 05 '16 at 23:04

Martijn

2,268
3
25
51

This discussion is also fairly interesting: https://github.com/sbt/sbt/issues/1001 – Martijn Jul 05 '16 at 23:40
Oh, right, thanks for pointing out the `(target in module).value`. As for threading the cleanup just to return it at the very end - I thought there might be a dedicated, maybe more elegant solution, but I'll take it. Thanks! In the end, it means that if I change the `stageCleanupTask`, then either `<<=` or `Def.sequential` should work. – Patryk Ćwiek Jul 06 '16 at 06:24
I take it back, `Def.sequential` still doesn't match all the types. – Patryk Ćwiek Jul 06 '16 at 06:27

score 3 · Accepted Answer · edited May 28 '18 at 21:31

3

Replace following code

stage := {
  val f = (stage in Universal).value
  stageCleanupTask.value
  f
}

By this one

import com.typesafe.sbt.packager.universal.UniversalPlugin.autoImport.Universal

stage := Def.taskDyn {
  val x = (stage in Universal).value
  Def.task {
    stageCleanupTask.value
    x
  }
}.value

That will lead to sequencing tasks and work as you expected.

edited May 28 '18 at 21:31

Neil

24,551
15
60
81

answered Mar 09 '18 at 09:05

kraken

484
7
18

How do you import `Universal` in this context? – Neil May 28 '18 at 21:12

Sequencing and overriding tasks in SBT

3 Answers3

Edit 1

Linked