0

Given a sorted RDD[String], how can I save it to file with a header?

val header: String = "header row"
val foo: RDD[(Int, String)] = ???
foo.coalesce(1).sortByKey().saveAsTextFile("//some/path")

This should save the data in order (I believe). How can I inject the header in the top line?

Synesso
  • 37,610
  • 35
  • 136
  • 207
  • What do you mean by "with a header"? column name? – giaosudau Aug 10 '16 at 02:11
  • No, just header info. It's not a CSV. The format is Metis input, which requires the first line to be a summary of the remaining data. – Synesso Aug 10 '16 at 02:20
  • For a similar question sean-owen suggested creating RDD for the header line, and performing union between both RDDs (header and original-RDD) http://stackoverflow.com/questions/26157456/add-a-header-before-text-file-on-save-in-spark – Yaron Aug 10 '16 at 08:40

0 Answers0