What I need to do is to save a Dataset to Hadoop, my code:
val df = spark.read.textFile("hdfs://path/to/file")
.flatMap(i => i.split("\\s+"))
.groupBy("value").count.toDF("word","count")
.write.json("hdfs://path/to/myjson.json")
but when I goto Hadoop, myjson.json is not a json file but a folder. When I read this folder with code like spark.read.json("hdfs://path/to/myjson.json").show again, it is exactly what I want.
My question is, how can I save a dataset to a file, not a folder ?
