Installing Spark and running it with Python

Asked Sep 26 '15 at 05:36

Active Sep 26 '15 at 14:45

Viewed 320 times

I'm alot more familiar with R than Python, but I heard its much easier to use Spark with Python than R, so thats why I'm trying to run Spark through Python. I'm using the Ninja IDE for Python

I downloaded the files for Spark-1.5.0 but now I don't know what to do. I went to http://spark.apache.org/docs/latest/building-spark.html. I'm using Windows 10 and used the command prompt to go my spark folder and then typed build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package. However, it then says build is not recognized as an internal or external command, operable program or batch file. Am I doing this right? Or is there another step I should be proceeding with?

edited Sep 26 '15 at 14:45

zero323

322,348
103
959
935

asked Sep 26 '15 at 05:36

CYPHER

You're in windows, so it's probably `build\mvn` unless you use cygwin. – Reactormonk Sep 26 '15 at 09:51
I got the same error if I try that – CYPHER Sep 26 '15 at 10:13
`build/mvn` is a Bash script. It won't work on directly on Windows. – zero323 Sep 26 '15 at 11:51
2

possible duplicate of [How to set up Spark on Windows?](http://stackoverflow.com/questions/25481325/how-to-set-up-spark-on-windows) – zero323 Sep 26 '15 at 11:51

Installing Spark and running it with Python

0 Answers0