How to avoid re-compiling generated source code

Question

Generating boilerplate source code with sbt works fine:

sourceGenerators in Compile <+= sourceManaged in Compile map { srcDir =>
  DslBoilerplate.generate(srcDir, Seq(
    "path/to/a/definition/file"
  ))
}

When I run sbt compile this also compiles the generated source code files, thus producing some class files. I just don't want the generated source code to be re-compiled every time I re-compile the project during development.

So, from the class files I made a jar file and used this instead of the generated source/class files (I deleted those). This worked fine, now having access to the generated code through the jar file. Is there a way though to let sbt do the 4 steps (if needed?) in the initial project build?:

1. generate source code files
2. compile those files
3. create a jar from the produced class files
4. delete source and class files

(In this question they use the sbt.IO.jar method to create a jar but there they already have existing files...)

Or is there another better approach than making a jar to avoid re-compiling generated source code?

---

Update 1 (see update 2 below)

Thanks, Seth, for your answer! It worked great to avoid generating the source files with each project compilation since the cache now remembers that they have been created. I'll certainly use this feature, thanks!

But this was actually not what I had in mind with my original question. Sorry for not being clear enough. It might be clearer if we think of this as 2 transformations happening:

Input file ---1---> Source file (*.scala) ---2---> Target file (*.class)

where the transformations are

1. generation of source code (from some information in an input file) and
2. compilation of the generated source code

This all works fine when I compile the project with sbt compile.

But then if I "rebuild the project" (in IntelliJ), the generated source code (from the sbt compilation) will compile again, and that's what I want to avoid - but at the same time have access to that code. Is there any other way to avoid compilation than placing this code in a jar and then delete the source and target files?

So I tried to continue along that line of thought wrestling with sbt to make it create a source and target jar - still can't make the target jar. This is what I came up with so far (with help from this):

sourceGenerators in Compile += Def.task[Seq[File]] {
  val srcDir = (sourceManaged in Compile).value
  val targetDir = (classDirectory in Compile).value

  // Picking up inputs for source generation
  val inputDirs = Seq("examples/src/main/scala/molecule/examples/seattle")

  // generate source files
  val srcFiles = DslBoilerplate.generate(srcDir, inputDirs)

  // prepare data to create jars
  val srcFilesData = files2TupleRec("", srcDir)
  val targetFilesData = files2TupleRec("", targetDir)

  // debug
  println("### srcDir: " + srcDir)
  println("### srcFilesData: \n" + srcFilesData.mkString("\n"))
  println("### targetDir: " + targetDir)
  println("### targetFilesData: \n" + targetFilesData.mkString("\n"))

  // Create jar from generated source files - works fine
  val srcJar = new File("lib/srcFiles.jar/")
  println("### sourceJar: " + srcJar)
  sbt.IO.jar(srcFilesData, srcJar, new java.util.jar.Manifest)

  // Create jar from target files compiled from generated source files
  // Oops - those haven't been created yet, so this jar becomes empty... :-(
  // Could we use dependsOn to have the source files compiled first?
  val targetJar = new File("lib/targetFiles.jar/")
  println("### targetJar: " + targetJar)
  sbt.IO.jar(targetFilesData, targetJar, new java.util.jar.Manifest)

  val cache = FileFunction.cached(
    streams.value.cacheDirectory / "filesCache",
    inStyle = FilesInfo.hash,
    outStyle = FilesInfo.hash
  ) {
    in: Set[File] => srcFiles.toSet
  }
  cache(srcFiles.toSet).toSeq
}.taskValue

def files2TupleRec(pathPrefix: String, dir: File): Seq[Tuple2[File, String]] = {
  sbt.IO.listFiles(dir) flatMap {
    f => {
      if (f.isFile && f.name.endsWith(".scala")) Seq((f, s"${pathPrefix}${f.getName}"))
      else files2TupleRec(s"${pathPrefix}${f.getName}/", f)
    }
  }
}

Maybe I still don't need to create jars? Maybe they shouldn't be created in the source generation task? I need help...

---

Update 2

Silly me!!! No wonder I can't make a jar with class files if I filter them with f.name.endsWith(".scala"), dohh

Since my initial question was not that clear, and Seth's answer is addressing an obvious interpretation, I'll accept his answer (after investigating more, I see that I should probably ask another question).

Answer 1

You want to use FileFunction.cached so that the source files aren't regenerated every time.

Here's an example from my own build:

Compile / sourceGenerators += Def.task[Seq[File]] {
  val src = (Compile / sourceManaged).value
  val base = baseDirectory.value
  val s = streams.value
  val cache =
    FileFunction.cached(s.cacheDirectory / "lexers", inStyle = FilesInfo.hash, outStyle = FilesInfo.hash) {
      in: Set[File] =>
        Set(flex(s.log.info(_), base, src, "ImportLexer"),
            flex(s.log.info(_), base, src, "TokenLexer"))
    }
  cache(Set(base / "project" / "flex" / "warning.txt",
            base / "project" / "flex" / "ImportLexer.flex",
            base / "project" / "flex" / "TokenLexer.flex")).toSeq
}.taskValue

Here the .txt and .flex files are input files to the generator. The actual work of generating the source files is farmed out to my flex method, which returns a java.io.File:

def flex(log: String => Unit, base: File, dir: File, kind: String): File =
  ...

You should be able to adapt this technique to your build.

FileFunction.cached is described in the API doc and in the sbt FAQ under "How can a task avoid redoing work if the input files are unchanged?" (http://www.scala-sbt.org/0.13/docs/Faq.html). (It would be nice if the material on caching was referenced from http://www.scala-sbt.org/0.13/docs/Howto-Generating-Files.html as well; currently it isn't.)