6

How do I do map and reduce operations using SparkR? All I can find is stuff about SQL queries. Is there a way to do map and reduce using SQL?

Matthew Jones
  • 944
  • 9
  • 17
  • Not sure what you mean by doing "map and reduce using SQL," but this this tutorial should help you get a sense of how to write map/reduce in SparkR: http://blog.rstudio.org/2015/05/28/sparkr-preview-by-vincent-warmerdam/ – sph21 Oct 09 '15 at 00:40

1 Answers1

3

See Writing R data frames returned from SparkR:::map for an example (the question itself). In short, the blog post referred to by sph21 is out of date. As of the current date, both map and reduce have been hidden in SparkR as private methods - there are open tickets to resolve that issue.

Community
  • 1
  • 1
russellpierce
  • 4,583
  • 2
  • 32
  • 44