Please tell me how to save the dataframe to a file with MY NAME in csv format. It may not work directly, but what workaround is there for this? We are talking about large volumes. So far, I can save it in a folder with my name, and the names part-0000..., but I would like to put it in a file with my name and maybe something more productive than repartition()
Asked
Active
Viewed 88 times
0
-
Hope this will help you https://stackoverflow.com/questions/41990086/specifying-the-filename-when-saving-a-dataframe-as-a-csv – dsk Jul 02 '21 at 07:13
-
You can coallesce to 1, then RDD[String] and foreachPartition. Use the Iterator[String] and use any IO api to create you file. note that you should include the header first. – Emiliano Martinez Jul 02 '21 at 07:23
-
Thanks so much. - dsk!!! – Dark Morning Jul 05 '21 at 15:45