0

I am trying to use the SparkR package in R. I have all dependent packages like devtools, Rtools.exe, etc.

When I try the following command:

install_github("amplab-extras/SparkR-pkg",subdir="pkg")

I get the following error:

Downloading github repo amplab-extras/SparkR-pkg@master
Error in function (type, msg, asError = TRUE ) :
  Received HTTP code 403 from proxy after CONNECT

To solve this I have set a working http_proxy, https_proxy but it is not working and throws above error. I am new to R/RStudio.

Phil
  • 7,287
  • 3
  • 36
  • 66
Umesh K
  • 13,436
  • 25
  • 87
  • 129
  • That package actually has shell scripts in it to download and install Spark, so needs to be run as root, and probably does not respect anything you set in R. – Neal Fultz Jun 02 '15 at 15:47
  • @Netal I am not setting proxy at R level I am setting as System env variables I think it should not be a problem no? – Umesh K Jun 02 '15 at 17:59
  • maybe, but it uses sbt for at least some of it. this may help for that part: http://stackoverflow.com/questions/13803459/how-to-use-sbt-from-behind-proxy – Neal Fultz Jun 02 '15 at 18:04

2 Answers2

1

I have installed SparkR on Windows 7, 64 bit with R-3.2.x and having Spark 1.4 installed on it.

** If you need to know about installing Spark on Windows, please check official documentation of Spark or step wise process listed here.

  • Go to bin folder of maven

    C:\Program Files\apache-maven-3.3.3\bin

  • Open notepad and paste the text

    "%~dp0\mvn.cmd" %*

  • Save the notepad in bin folder as mvn.bat as shown below

    C:\Program Files\apache-maven-3.3.3\bin\mvn.bat

  • Restart Rstudio and execute

    library(devtools) install_github("repo/SparkR-pkg", ref="branchname", subdir="pkg")

Kamlesh
  • 26
  • 2
0

Since SparkR was merged into the Spark Core project starting in the 1.4 release in June 2015, right now the SparkR build instructions are same as the Spark build instructions. Make sure you add the maven flag -Psparkr for SparkR.

As an example, you can build a version of Spark with SparkR as follows:

build/mvn -Psparkr -DskipTests clean package

For more details, please see: http://spark.apache.org/docs/latest/building-spark.html.

Emaasit
  • 31
  • 1
  • 7