1

My question is almost same as Add a header before text file on save in Spark. The difference is that my header RDD is 

String headerSTR = "inc_id,po_id,ass,inci_type,cat,sub_cat";
JavaRDD<String> PMheader = jsc.parallelize(Arrays.asList(headerSTR));

And my lines RDD is of PM Table type.

JavaRDD<PMTable>rdd_records=noheader.map(new Function<String,PMTable>(){---
PMTable sd = new PMTable(----);
return sd;});
rdd_records.saveAsTextFile();
mergeAllFiles();

I have merged all the result files to a single csv file which does not contain header .Now I need to get union of header rdd and lines rdd .But the method union(JavaRDD) in the type JavaRDD is not applicable for the arguments (JavaRDD of PMTable type). So how can i get the union of header and lines using spark-java api.

Thanks in advance.

Community
  • 1
  • 1
Anonymous
  • 337
  • 2
  • 5
  • 12
  • Now I am able to get the union of both headers and lines.I have created a headerRDD of < PMTable> type. And take the union of both. – Anonymous Dec 30 '16 at 04:07

0 Answers0