0

I search a simple way of writing a scala dataset into a json file. I tried several ways like:

mydataset.write.json("selected_games.json")

or

mydataset.coalesce(1).write.mode("overwrite").json("selected_games.json")

but it always creates a directory named "selected_games.json" with four files inside:

._SUCCESS.crc
.part-00000-59e33267-59a4-43dd-b7b0-6e3805b72160-c000.json.crc
_SUCCESS
part-00000-59e33267-59a4-43dd-b7b0-6e3805b72160-c000.json

how to have a single simple json file instead of this mess ?

Frédéric Methel
  • 202
  • 1
  • 2
  • 1
    Refer this - https://stackoverflow.com/questions/41990086/specifying-the-filename-when-saving-a-dataframe-as-a-csv and https://stackoverflow.com/questions/40792434/spark-dataframe-save-in-single-file-on-hdfs-location – Vikramsinh Shinde Aug 16 '23 at 07:48

0 Answers0