0

I work on spark application using (spark 2.0.0 & scala 2.11.8) and the application works fine within intellij Idea environment. I've extracted application as jar file and tried to run spark application from jar file but this error raised on terminal:

Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;
at org.apache.spark.util.Utils$.getSystemProperties(Utils.scala:1632)
at org.apache.spark.SparkConf.loadFromSystemProperties(SparkConf.scala:65)
at org.apache.spark.SparkConf.<init>(SparkConf.scala:60)
at org.apache.spark.SparkConf.<init>(SparkConf.scala:55)
at Main$.main(Main.scala:26)
at Main.main(Main.scala)

I've read discussions and similar question but all of them talk about different scala versions, however my sbt file is this:

name := "BaiscFM"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.0"
libraryDependencies += "org.apache.spark" % "spark-streaming_2.11" % "2.0.0"
libraryDependencies += "org.apache.spark" % "spark-streaming-kafka-0-8_2.11" % "2.0.0"
libraryDependencies += "com.datastax.spark" % "spark-cassandra-connector_2.11" % "2.0.0"
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.0"
libraryDependencies += "org.apache.spark" % "spark-graphx_2.11" % "2.0.0"
libraryDependencies += "com.typesafe.akka" % "akka-actor_2.11" % "2.4.17"
libraryDependencies += "net.liftweb" % "lift-json_2.11" % "2.6"
libraryDependencies += "com.typesafe.play" % "play-json_2.11" % "2.4.0-M2"
libraryDependencies += "org.json" % "json" % "20090211"
libraryDependencies += "org.scalaj" % "scalaj-http_2.11" % "2.3.0"
libraryDependencies += "org.drools" % "drools-core" % "6.3.0.Final"
libraryDependencies += "org.drools" % "drools-compiler" % "6.3.0.Final"

How to fix this problem?

mkrieger1
  • 19,194
  • 5
  • 54
  • 65
Luay AL-Assadi
  • 426
  • 7
  • 16
  • How exactly do you build the jar? Did you verify the jar contains everything needed to run it? In some cases, when multiple jars are combined into one, it may not work because of the conflicts. – CrazyCoder Apr 03 '17 at 08:47
  • 2
    Possible duplicate of [Spark Submit fails with java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;](http://stackoverflow.com/questions/30342273/spark-submit-fails-with-java-lang-nosuchmethoderror-scala-predef-conformsls) – T. Gawęda Apr 03 '17 at 11:13
  • 1
    Your Scala version on cluster is not correct – T. Gawęda Apr 03 '17 at 11:13
  • I've try to build jar with dependency linked by mainfest, sorry but what do you mean with (Scala version on cluster is not correct)? scala version on claster is the same 2.11.8 – Luay AL-Assadi Apr 03 '17 at 12:50
  • btw the mentioned question is not similar to my problem, I've the compatible scala version with spark and cassandra. however; the mention question try to run incompatible scala version with spark. – Luay AL-Assadi Apr 04 '17 at 15:13

0 Answers0