11

I have a Scala application which processes binary files from some directory in resources. I would like to get this directory as java.io.File and list all the contents. In the newest sbt I am unable to do it the straight way.

I have created minimal repo with my issue:

https://github.com/mat646/sbt-resource-bug

The issue does not occur with sbt 0.13.18 and lower. So after some research I've found out that since sbt 1.0 the design has changed and such issue has been already addressed here:

https://github.com/sbt/sbt/issues/3963

So the offered solutions were:

  1. downgrade to sbt 0.13 (which I would like to avoid)
  2. extracting project jar itself (which I found pretty nagging, as I still haven't solved it yet - the https://github.com/sbt/io contains gunzip method but for me it still fails to extract my directory from jar, but here I might be misunderstanding how to extract nested file from project jar)
    (so as the sbt 1.0+ works on build jar for reading files getResourceAsStream works perfectly but fails for my issue)
Tomer Shetah
  • 8,413
  • 7
  • 27
  • 35
mtsokol
  • 170
  • 1
  • 12

1 Answers1

13

run task seems to have been rewired in Forward run task to bgRun #3477 and one side-effect was to use packaged jars on classpaths instead of class directories, so that getClass.getResource("/my-dir") returns

jar:file:/var/folders/84/hm7trc012j19rbgtn2h4fg6c0000gp/T/sbt_1397efb8/job-1/target/43a04671/sbt-resource-bug_2.12-0.1.jar!/my-dir

instead of

file:/Users/amani/IdeaProjects/sbt-resource-bug/target/scala-2.12/classes/my-dir

Workaround 1 : Redefine run to the old behaviour

As a workaround, we could try reverting the rewiring in our build.sbt like so:

run := Defaults.runTask(fullClasspath in Runtime, mainClass in run in Compile, runner in run).evaluated

Now File.listFiles should work, for example,

new File(getClass.getResource("/my-dir").getFile).listFiles().foreach(println)

should output

sbt:sbt-resource-bug> run
[info] Running Main 
/Users/mario/IdeaProjects/sbt-resource-bug/target/scala-2.12/classes/my-dir/file2.txt
/Users/mario/IdeaProjects/sbt-resource-bug/target/scala-2.12/classes/my-dir/file1.txt

 Workaround 2: Use JarFile to work with JARs directly

Alternatively, if we wish to keep the current rewiring, JarFile can be used to list the contents of JAR files. For example, given

object ListFileNamesInJarDirectory {
  def apply(dir: String): List[String] = {
    import scala.collection.JavaConverters.enumerationAsScalaIteratorConverter
    val jar = new File(getClass.getProtectionDomain().getCodeSource().getLocation().getPath())
    (new JarFile(jar))
      .entries()
      .asScala
      .toList
      .filter(!_.isDirectory)
      .filter(entry => entry.getRealName.contains(dir))
      .map(_.getName)
  }
}

then

ListFileNamesInJarDirectory("my-dir").foreach(println)

should output

my-dir/file1.txt
my-dir/file2.txt

Afterwards, getResourceAsStream can be used to get at the actual files in the jar. Note how we get File to represent the jar file:

val jar = new File(getClass.getProtectionDomain().getCodeSource().getLocation().getPath())
Community
  • 1
  • 1
Mario Galic
  • 47,285
  • 6
  • 56
  • 98
  • Thank you so much for such comprehensive answer! This precisely solves my issue! Both workarounds work seamlessly, but could you give additional opinion on these solutions? Second method is clear to me and it definitely avoids unpacking whole project jar to some tmp/ directory as some users on github issue suggested. In first one I'm unfamiliar with sbt internals. So are there any costs (performance, safety) which I might bear due to such rewiring? Or can I losslessly use it? – mtsokol Apr 06 '19 at 09:41