compiling and running jars with scalac vs sbt?

Question

I am trying to compile a fat jar for my Scala project that contains all of my dependencies according to this stackoverflow post, so that I can use it in a Java application. I am new to Scala/Java/JVM, so please be patient. I am using IntelliJ and scala 2.12.4, however I am running my sbt commands from the OS X Terminal.

To start, I am able to run sbt assembly and get a padsystem-assembly-0.0.1.jar in /target/scala-2.12/. (I had to make an assembly.sbt and modify my build.sbt with a "Merge Strategy" to get it to work.) However when I try to run this jar with scala

computer: dir user$ scala target/scala-2.12/padsystem-assembly-0.0.1.jar
scala target/scala-2.12/padsystem-assembly-0.0.1.jar 
java.lang.NullPointerException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at scala.reflect.internal.util.ScalaClassLoader.$anonfun$tryClass$1(ScalaClassLoader.scala:45)
    at scala.util.control.Exception$Catch.$anonfun$opt$1(Exception.scala:242)
    at scala.util.control.Exception$Catch.apply(Exception.scala:224)
    at scala.util.control.Exception$Catch.opt(Exception.scala:242)
    at scala.reflect.internal.util.ScalaClassLoader.tryClass(ScalaClassLoader.scala:45)
    at scala.reflect.internal.util.ScalaClassLoader.tryToInitializeClass(ScalaClassLoader.scala:41)
    at scala.reflect.internal.util.ScalaClassLoader.tryToInitializeClass$(ScalaClassLoader.scala:41)
    at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.tryToInitializeClass(ScalaClassLoader.scala:125)
    at scala.reflect.internal.util.ScalaClassLoader.run(ScalaClassLoader.scala:92)
    at scala.reflect.internal.util.ScalaClassLoader.run$(ScalaClassLoader.scala:91)
    at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:125)
    at scala.tools.nsc.CommonRunner.run(ObjectRunner.scala:22)
    at scala.tools.nsc.CommonRunner.run$(ObjectRunner.scala:21)
    at scala.tools.nsc.JarRunner$.run(MainGenericRunner.scala:14)
    at scala.tools.nsc.CommonRunner.runAndCatch(ObjectRunner.scala:29)
    at scala.tools.nsc.CommonRunner.runAndCatch$(ObjectRunner.scala:28)
    at scala.tools.nsc.JarRunner$.runAndCatch(MainGenericRunner.scala:14)
    at scala.tools.nsc.JarRunner$.runJar(MainGenericRunner.scala:26)
    at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:72)
    at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:85)
    at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:96)
    at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:101)
    at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

And when I try to run it with java (which is my main goal):

computer: dir user$ java target/scala-2.12/padsystem-assembly-0.0.1.jar 
Error: Could not find or load main class target.scala-2.12.padsystem-assembly-0.0.1.jar

Now, before you ask, "Are you sure your code is working?", I will note that I am able to sbt run my code successfully, and I can also sbt compile and sbt package it successfully as well.

Interestingly, if I try to sbt compile src/main/scala/OdinExtractor.scala from the base directory:

sbt compile src/main/scala/OdinExtractor.scala
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/user/.sbt/1.0/plugins
[info] Loading project definition from /Users/user/PAD_IE/project/project
[info] Loading settings from assembly.sbt,plugins.sbt ...
[info] Loading project definition from /Users/user/PAD_IE/project
[info] Loading settings from build.sbt ...
[info] Set current project to padsystem (in build file:/Users/user/PAD_IE/)
[info] Executing in batch mode. For better performance use sbt's shell
[success] Total time: 4 s, completed Mar 16, 2018 3:35:49 PM
[error] Expected ID character
[error] Not a valid command: src (similar: set)
[error] Expected project ID
[error] Expected configuration
[error] Expected ':' (if selecting a configuration)
[error] Expected key
[error] Not a valid key: src (similar: sources, ps, run)
[error] src/main/scala/OdinExtractor.scala
[error]  ^

OR if I try to scalac src/main/scala/OdinExtractor.scala or if I cd into the src/main/scala dir and try to run sbt compile OdinExtractor.scala I get:

[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/user/.sbt/1.0/plugins
[info] Updating ProjectRef(uri("file:/Users/user/.sbt/1.0/plugins/"), "global-plugins")...
[info] Done updating.
[info] Set current project to scala (in build file:/Users/user/PAD_IE/src/main/scala/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] Compiling 1 Scala source to /Users/user/PAD_IE/src/main/scala/target/scala-2.12/classes ...
[error] /Users/user/PAD_IE/src/main/scala/OdinExtractor.scala:3:12: object clulab is not a member of package org
[error] import org.clulab.odin.Mention
[error]            ^
[error] /Users/user/PAD_IE/src/main/scala/OdinExtractor.scala:4:12: object clulab is not a member of package org
[error] import org.clulab.processors.Document
[error]            ^
[error] /Users/user/PAD_IE/src/main/scala/OdinExtractor.scala:5:12: object vinci is not a member of package org
[error] import org.vinci.pad.padsystem.PadSystem
[error]            ^
[error] three errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed Mar 16, 2018 3:38:06 PM

Why would I be able to sbt run my code successfully, but not be able to run the jar with scala or java, especially when all of the dependencies are included in the fat jar?

Why do I get errors with my depdencies when I do scalac src/main/scala/OdinExtractor.scala and sbt compile OdinExtractor.scala? The first I am guessing because scalac wants something like

scalac -cp "all:of:the:classpath:stuff:ever" OdinExtractor.scala.

(By the way, I can't figure out how to do this...)

Which, as I understand, is why we use sbt to begin with, yes? To avoid the messy classpath stuff. Which brings me to my next question: Why does sbt compile fail when I point it at the specific file? And why does it fail when I run it from the src/main/scala directory? I don't know about the first one, but my guess for the second question is that sbt must always be run from the base directory?

Back to the main goal: producing a fat jar that I can run with Java... Does anyone have any idea how I can debug this? I don't understand why the assembly jar fails, but the code still runs with sbt run. I originally thought the best way to debug this would be to just compile the object with def main(), and try to run it with both Scala and Java, but this has proven very troublesome.

In the end, I have an eerie feeling that perhaps all of this boils down to either 1) my confusion about classpaths, or 2) perhaps my project structure? But since my stuff runs with sbt run, I'm just at a total loss... Please help! Let me know if there's anything else I need to add to my post to make it more clear. Thanks!

Edit - Per my build.sbt I did include the scala-library jar, so that it can be parsed by Java. I.e. "org.scala-lang" % "scala-library" % "2.12.4"

Also, if its of any importance, my Object has a def main instead of extends App. Not sure if that matters...

Add your `assembly.sbt` to the question, it may help. When you run it with java, use `java -jar target/...` or `java -cp target/... your.main.Class`. And `sbt compile src/...` won't (isn't supposed to) work on a separate source file. — laughedelic, Mar 17 '18 at 02:44

score 0 · Accepted Answer · answered Mar 18 '18 at 19:30

The answer ended up being with the build.sbt. I found the answer in this stackoverflow post. I had to add

mainClass in assembly := Some("NameOfMyMainClass")

into the build.sbt.

After that, I did

sbt clean assembly

and was able to run both java -jar target/.../my-fat-jar.jar and scala target/.../my-fat-jar.jar.

Shoutout to @laughedelic for pointing out that I should use java -jar, and for answering my question about sbt compile src/... :')

compiling and running jars with scalac vs sbt?

1 Answers1