error execution spark with intellij

Question

when i try to run spark with my intellij.

   Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
        at org.apache.spark.util.Utils$.getCallSite(Utils.scala:1306)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
        at batch.BatchT$.main(BatchT.scala:15)
        at batch.BatchT.main(BatchT.scala)

I use spark 1.6, and i create a module with scala 2.12.04 support.

https://stackoverflow.com/questions/75947449/run-a-scala-code-jar-appear-nosuchmethoderrorscala-predef-refarrayops — Dmytro Mitin, Apr 07 '23 at 04:58

score 4 · Accepted Answer · answered Feb 05 '18 at 09:35

The reason for this is that scala is not compatible between minor versions. Spark 1.6 is compiled with scala 2.10 by default (an option exists to manually compile it with scala 2.11). Spark 2.0+ is compiled with scala 2.11 by default.

No spark version is compiled with scala 2.12 currently (support for this is planned only for spark 3.0)

The easiest solution would be to downgrade your scala version to 2.10 (or 2.11 if you use a newer spark version such as 2.0+ or a specifically compiled spark 1.6).

error execution spark with intellij

1 Answers1