3

I have a dataframe like below:

+---+-----+-------------------+
|  x|    y|                  z|
+---+-----+-------------------+
|xyz|12223|A,123@B,456@C,98765|
|abc|12456|              A,123|
+---+-----+-------------------+

I save it as csv:

index.write.csv("D:\\spark\\tmp\\dd2")

two issues here:

  1. Would you like tell me how to save the column name x,y,z to the header of csv please?
  2. How can I save the output to a single file please?
Alper t. Turker
  • 34,230
  • 9
  • 83
  • 115
Robin
  • 695
  • 1
  • 7
  • 10

1 Answers1

4

Would you like tell me how to save the column name x,y,z to the header of csv please?

You have to use header option:

index.write.option("header", "true").write(path)

How can I save the output to a single file please?

Write single CSV file using spark-csv

Alper t. Turker
  • 34,230
  • 9
  • 83
  • 115