3

I'm trying to install SparkR

library(devtools)
install_github("amplab-extras/SparkR-pkg", subdir="pkg")

but then I find this error

Launching sbt from sbt/sbt-launch-0.13.6.jar Invalid or corrupt jarfile sbt/sbt-launch-0.13.6.jar make: *** [target/scala-2.10/sparkr-assembly-0.1.jar] Error 1 ERROR: compilation failed for package ‘SparkR’ * removing ‘/Users/apple/Library/R/3.2/library/SparkR’ Error: Command failed (1)

zero323
  • 322,348
  • 103
  • 959
  • 935
Lucia
  • 615
  • 1
  • 9
  • 16
  • this post was given and mentioned that the problem is with "Make" , may help you solve it https://support.rstudio.com/hc/en-us/community/posts/205133157-ERROR-compilation-failed-for-package-SparkR- –  Feb 28 '16 at 12:21
  • Out of curiosity, why do you need this? – zero323 Feb 28 '16 at 15:42
  • because running big data with R is a pain of A – Lucia Mar 01 '16 at 02:41

1 Answers1

1

Read before proceeding:

amplab-extras/SparkR-pkg is no longer maintained. Current versions of SparkR are shipped with Spark itself. See also Installing of SparkR

Sbt download links in the repository are invalid and what you get is actually a HTML file. You can either correct URL2 in pkg/src/sbt/sbt so it points to:

http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.6/sbt-launch.jar

or download and install sbt, clone the repository:

git clone https://github.com/amplab-extras/SparkR-pkg.git

go to src:

cd SparkR-pkg/pkg/src

assembly:

sbt assembly

and install:

R -e "devtools::install('.')"
Community
  • 1
  • 1
zero323
  • 322,348
  • 103
  • 959
  • 935
  • http://stackoverflow.com/questions/31184918/installing-of-sparkr this works very well. – Lucia Mar 01 '16 at 02:51
  • Glad to hear that. I thought you may be looking for a new version after all. – zero323 Mar 01 '16 at 03:08
  • then I'm trying to create a spark context by > sc <- sparkR.init(master="local[*]") Error: could not find function "sparkR.init" – Lucia Mar 01 '16 at 03:18
  • And how did you attach the library? – zero323 Mar 01 '16 at 03:20
  • I just follow this link http://ampcamp.berkeley.edu/5/exercises/sparkr.html things works fine until I create the spark context – Lucia Mar 01 '16 at 03:24
  • This exercises are no longer applicable in Spark 1.4+. Moreover it doesn't environment setup. SparkR is no standalone package. You can start with https://spark.apache.org/docs/latest/sparkr.html – zero323 Mar 01 '16 at 03:34
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/104946/discussion-between-lucia-and-zero323). – Lucia Mar 01 '16 at 03:38