2

I am new to Spark, I am trying to start master manually (using MINGW64 in windows 10).

When I do this, ~/Downloads/spark-1.5.1-bin-hadoop2.4/spark-1.5.1-bin-hadoop2.4/sbin $ ./start-master.sh

I got these logs,

ps: unknown option -- o
Try `ps --help' for more information.
starting org.apache.spark.deploy.master.Master, logging to /c/Users/Raunak/Downloads/spark-1.5.1-bin-hadoop2.4/spark-1.5.1-bin-hadoop2.4/sbin/../logs/spark--org.apache.spark.deploy.master.Master-1-RINKU-CISPL.out
ps: unknown option -- o
Try `ps --help' for more information.
**failed to launch org.apache.spark.deploy.master.Master:**
  Spark Command: C:\Program Files\Java\jre1.8.0_77\bin\java -cp C:/Users/Raunak/Downloads/spark-1.5.1-bin-hadoop2.4/spark-1.5.1-bin-hadoop2.4/sbin/../conf\;C:/Users/Raunak/Downloads/spark-1.5.1-bin-hadoop2.4/spark-1.5.1-bin-hadoop2.4/lib/spark-assembly-1.5.1-hadoop2.4.0.jar;C:\Users\Raunak\Downloads\spark-1.5.1-bin-hadoop2.4\spark-1.5.1-bin-hadoop2.4\lib\datanucleus-api-jdo-3.2.6.jar;C:\Users\Raunak\Downloads\spark-1.5.1-bin-hadoop2.4\spark-1.5.1-bin-hadoop2.4\lib\datanucleus-core-3.2.10.jar;C:\Users\Raunak\Downloads\spark-1.5.1-bin-hadoop2.4\spark-1.5.1-bin-hadoop2.4\lib\datanucleus-rdbms-3.2.9.jar -Xms1g -Xmx1g org.apache.spark.deploy.master.Master --ip RINKU-CISPL --port 7077 --webui-port 8080

What am I doing wrong , Should I have to configure Hadoop package also for Spark?

Amit Kumar
  • 2,685
  • 2
  • 37
  • 72
rinku buragohain
  • 329
  • 2
  • 7
  • 15

1 Answers1

2

Just found answer here: https://spark.apache.org/docs/1.2.0/spark-standalone.html

"Note: The launch scripts do not currently support Windows. To run a Spark cluster on Windows, start the master and workers by hand."

I think windows is not a good choice for Spark, anyway good luck!

Spider
  • 1,380
  • 4
  • 22
  • 42
  • 1
    try spark-class org.apache.spark.deploy.master.Master. For workers spark-class org.apache.spark.deploy.worker.Worker spark://MASTER_IP:MASTER_PORT – Rdesmond Jul 12 '16 at 06:34