0

I am a noob to Spark and Intellij.I want to run Spark using Scala I initially installed Scala 2.12 and created the SBT accordingly.Then I got a NoSuchMethod runtime error on

       sc =  new SparkContext(conf)

From the solution posted NoSuchMethodError when using Sparka and IntelliJ I used Scala version 2.11.3 while creating the project and used the SBT

version := "0.1"

scalaVersion := "2.11.3"

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.2"

I am now getting the error 

   Error:scalac: No 'scala-library*.jar' in Scala compiler classpath in Scala SDK SBT: org.scala-lang:scala-library:2.11.3:jar

This is the library on the External libraries section Snapshot from Project External Libraries

I tried creating the project from scratch and Cache Invalidate/Restart option.Same result

I also tried downloading via Maven through File -> Project Structure.Only found spark-core 2.10.Showed the same NoSuchMethod Error

Can anyone identify the problem?

Solo
  • 193
  • 2
  • 12
  • can you see scala-compiler.jar, scala-library.jar, scala-reflect.jar libraries in Project Structure -> modules ? – koiralo Oct 11 '17 at 12:16
  • Thanks @Shankar Koirala.I could not see them before but I just uninstalled scala plugin and restarted and reinstalled and it worked.It was just magic..:) – Solo Oct 11 '17 at 12:36

1 Answers1

1

It clearly says that, It could not find scala-library*.jar file

So go to 

"Project Structure -> Modules"

And see these jar files 

scala-compiler.jar, scala-library.jar, scala-reflect.jar

If they are absent from the Modules add them manually or Reinstall the scala and scala plugin.

Hope this should work!

koiralo
  • 22,594
  • 6
  • 51
  • 72