1

Following the question here: Unable to launch SparkR in RStudio

I encountered problem using SparkR. I used the following:

> Sys.setenv(SPARK_HOME='C:/Users/aw/Downloads/spark-1.6.0-bin-hadoop2.6/spark-1.6.0-bin-hadoop2.6/bin')
> .libPaths(c(file.path(Sys.getenv('SPARK_HOME'), 'R', 'lib'), .libPaths()))
> sc=sparkR.init(master="local")

Launching java with spark-submit command C:/Users/aw/Downloads/spark-1.6.0-bin-hadoop2.6/spark-1.6.0-bin-hadoop2.6/bin/bin/spark-submit.cmd   sparkr-shell C:\Users\aw\AppData\Local\Temp\Rtmp8CCseT\backend_port1cc4622cb87 
Error in sparkR.init(master = "local") : 
  JVM is not ready after 10 seconds
In addition: Warning message:
running command '"C:/Users/aw/Downloads/spark-1.6.0-bin-hadoop2.6/spark-1.6.0-bin-hadoop2.6/bin/bin/spark-submit.cmd"   sparkr-shell C:\Users\aw\AppData\Local\Temp\Rtmp8CCseT\backend_port1cc4622cb87' had status 127

We can see that I run the SPARK from the shell: enter image description here

enter image description here enter image description here enter image description here

I use Window 10 and R version 3.2.1 Patched (2015-07-16 r68681).

Community
  • 1
  • 1
Avi
  • 2,247
  • 4
  • 30
  • 52
  • Thanks, I added the results to the question @DmitriChubarov Can you please look at the results? – Avi Mar 21 '16 at 11:56
  • `JVM is not ready after 10 seconds` can occur sometimes due to heap memory issues. try setting ` Sys.setenv(SPARK_MEM='512m')` and try initializing spark. – Vikram Venkat Mar 21 '16 at 12:13
  • Did and got error: > sc=sparkR.init(master="local") Launching java with spark-submit command C:/Users/aw/Downloads/spark-1.6.0-bin-hadoop2.6/spark-1.6.0-bin-hadoop2.6/bin/bin/spark-submit.cmd sparkr-shell C:\Users\aw\AppData\Local\Temp\Rtmp8CCseT\backend_port1cc41e0d7eed Error in sparkR.init(master = "local") : JVM is not ready after 10 seconds In addition: Warning message: running command '"C:/Users/aw/Downloads/spark-1.6.0-bin-hadoop2.6/spark-1.6.0-bin-hadoop2.6/bin/bin/spark-submit.cmd" sparkr-shell C:\Users\aw\AppData\Local\Temp\Rtmp8CCseT\backend_port1cc41e0d7eed' had status 127 – Avi Mar 21 '16 at 12:23
  • 1
    The PATH values are identical up to the prepended R-3.2.1 path in the first (error) case. So, something else is causing this `running command ... has status 127`. – Dima Chubarov Mar 21 '16 at 12:44

0 Answers0