0

I would like to run Spark from source code on my Windows machine. I did the following steps:

  1. git clone https://github.com/apache/spark
  2. Added the SPARK_HOME variable into the user variables.
  3. Added %SPARK_HOME%\bin to the PATH variable.
  4. ./build/mvn -DskipTests clean package
  5. ./bin/spark-shell

The last command returns the following error: enter image description here

What should I do to fix the error?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Sergei Russkikh
  • 123
  • 1
  • 7

1 Answers1

2

First, refer to the link below for the solution. The top voted answer gave me the working script for this problem.
: Failed to start master for Spark in Windows

The reason is that spark launch scripts do not support Windows. The spark documentation (https://spark.apache.org/docs/1.2.0/spark-standalone.html) insists you to start the master and workers manually if you are a Windows user. So you need to first run the master and then run spark-shell.

sjyang
  • 36
  • 1