Create a new RDD with wholeTextFiles in Spark

Question

I am trying to use wholeTextFiles of spark to read the directory, my file RDD contains (String,String) where first String is my filename and second my contents of the file.

I want to map this RDD to another one with just the contents of my file, how can I do it?

Thanks!

val file = sc.wholeTextFiles("./Desktop/093")

file.first
res0: (String, String) = 
(file:/Users/Desktop/093/nc-no-na.clusters.093.001.txt,"199 197 5   5   168 0   0.932125    11101111000000110100000000000000000000000000001010100000011100001000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001101101111100000000000000000011100000000000000000000000000100000111011000000000000000000000000000000000000000000000000000000000000000011110010111001001110000000011100000000010000000000000000000000000010000000000000000000000000000000000000000011111111111101010111000000000000000000000000000000000000000000000000000000000000000001100000000000000000000000000000000000000000101110101110101011010000000000000000001100001100000011110000000000000000000011111011110011100...

score 0 · Accepted Answer · answered Dec 02 '15 at 03:08

0

For example like this:

import org.apache.spark.rdd.RDD

val content: RDD[String] = file.map(_._2)

answered Dec 02 '15 at 03:08

zero323

322,348
103
959
935

[What are all the uses of an underscore in Scala?](http://stackoverflow.com/q/8000903/1560062), http://www.scala-lang.org/api/current/index.html#scala.Tuple2@_2:T2 – zero323 Dec 26 '15 at 04:56

Create a new RDD with wholeTextFiles in Spark

1 Answers1