0

I made a jar file for a Kafka Streams application coded in Scala using both IntelliJ IDEA itself and then SBT. Following are the problems I have been facing and so far I haven't succeeded:

First, I made a jar file using IntelliJ IDEA. Upon running the Jar file, it gave the following error:

java.lang.ClassNotFoundException: Main ...

Following are the steps I took to make the Jar file in IntelliJ IDEA:

  • Selected Project Structure from the file menu.
  • Went into the 'Artifacts' tab.
  • Clicked on the + button.
  • Selected the main class and the META-INF directory location
  • Added all the Scala libraries and dependencies (and made sure the META-INF dir was included)
  • Applied the settings and built the artifact.
  • Run the Jar using Scala command. Gave the aforementioned error.

After that I tried making the Jar file using SBT. I made an assembly.sbt file inside the project directory and then I reimported the project. After that, I executed the following commands:

  • sbt clean
  • sbt package
  • sbt assembly

It made a Jar file. This time, the problem was that it wasn't picking some of the Kafka libraries randomly. Sometimes, it'll throw, for example, that KafkaStreams not found. Then I'll try to rebuild the jar and the new jar would then throw StreamsBuilder not found. I kept on trying and to sum up, following are some of the errors I remember:

NoClassDefFoundException ..../KafkaStreams
NoClassDefFoundException ..../StreamsBuilder etc.

Here '...' means the full package name/path.

build.sbt:

name := "StreamsNew"

version := "0.1"

scalaVersion := "2.12.7"

mainClass := Some("src/main/scala/StreamsDemo/Main.scala")

libraryDependencies ++= Seq(
 //"org.slf4j" % "slf4j-simple" % "1.7.28",
  "org.apache.kafka" % "kafka-streams" % "2.3.0",
  "org.apache.kafka" %% "kafka-streams-scala" % "2.3.0",
)

assembly.sbt:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.10" )
resolvers += Resolver.url("bintray-sbt-plugins", url("http://dl.bintray.com/sbt/sbt-plugin-releases"))(Resolver.ivyStylePatterns)

Scala version: 2.12.7
SBT Version: 1.6.3

Cyber Knight
  • 184
  • 2
  • 10
  • Can you just run the App in IntelliJ alone? While there is nothing wrong with Scala, are you required to use it? Plenty of getting started examples for Java w/ Gradle out there. Kotlin could reduce the verbosity for you. SpringBoot can help you get an executable program out quicker – OneCricketeer Jan 23 '20 at 05:40
  • 1
    @cricket_007 yes, it is running in IntelliJ IDEA absolutely fine. And using Scala is a client requirement, that's why. – Cyber Knight Jan 23 '20 at 05:51
  • And sbt as well, I guess? Are you able to share any code? – OneCricketeer Jan 23 '20 at 05:53
  • 1
    You can run [`jar tf file.jar | grep kafka`](https://docs.oracle.com/javase/tutorial/deployment/jar/view.html) to see what **is** in the package – OneCricketeer Jan 23 '20 at 05:54
  • @cricket_007 Actually, I just used SBT to make the jar file after the Jar failed to run properly when made through IntelliJ. I wasn't using SBT much before (being new to Scala and such). And gonna try the command you posted above. – Cyber Knight Jan 23 '20 at 05:57
  • If the client only requires Scala, you can still compile it using Maven or Gradle. If you want help with sbt-assembly, then please post the code – OneCricketeer Jan 23 '20 at 05:59
  • @cricket_007 you mean the code written in assembly.sbt and build.sbt? – Cyber Knight Jan 23 '20 at 06:03
  • 1
    Please [edit] the question and format the code appopriately – OneCricketeer Jan 23 '20 at 06:17
  • @cricket_007 edited the code in question. – Cyber Knight Jan 23 '20 at 06:21
  • I don't know much about sbt, but does this help? https://stackoverflow.com/questions/28459333/how-to-build-an-uber-jar-fat-jar-using-sbt-within-intellij-idea – OneCricketeer Jan 23 '20 at 06:38
  • @cricket_007 it is somewhat helpful. In fact, I came across it before as well. I'll see what I can do with this link. I guess I need to invest time in properly learning SBT. Thanks for your time, mate! – Cyber Knight Jan 23 '20 at 06:41
  • If anything, I think the mainClass should be `StreamsDemo.Main`, assuming `package StreamsDemo; object Main extends App{}` – OneCricketeer Jan 23 '20 at 06:43
  • You are right. It is like that. @cricket_007. But still the issue persists. Maybe I should look at Scala/SBT version combinations. – Cyber Knight Jan 23 '20 at 06:58

1 Answers1

0

You don't need to use sbt package, because you want a fat jar (for the dependencies). For that, you only need to execute sbt clean and sbt assembly. More so, the main class path should only contain StreamsDemo.Main instead of the whole path. Now, try running the jar made through assembly. Also, you don't need the resolvers in the assembly.sbt.

Mujtaba Faizi
  • 284
  • 1
  • 7
  • 21