Create json file from dataset in spark scala

Question

I have the following dataset:

+------+-----------+-------+----------+
|userID|   A       |B      |  C       |
+------+-----------+-------+----------+
|1     |242.0      |NULL   |7         |
|2     |NULL       |3.0    |5.0       |
|3     |NULL       |70.0   |17.0      |

I want to transform it into the following json format:

"result":
[
{"userID": 1, "A": 242.0, "B": NULL, "C": 7 }, 
{"userID": 2, "A": NULL, "B": 3.0, "C": 5.0 }, 
{"userID": 3, "A": NULL, "B": 70.0, "C": 17.0 }, 
]

I appreciate help with that.

Possible duplicate of [How to convert Row to json in Spark 2 Scala](https://stackoverflow.com/questions/41601874/how-to-convert-row-to-json-in-spark-2-scala) — Rahul Sharma, Jun 17 '18 at 18:59
Possible duplicate of [DataFrame to Json Array in Spark](https://stackoverflow.com/questions/45141929/dataframe-to-json-array-in-spark) — Ramesh Maharjan, Jun 18 '18 at 03:21

score 0 · Answer 1 · answered Jun 18 '18 at 08:42

0

myDataset.write.json("/tmp/my_dir/")

answered Jun 18 '18 at 08:42

Elior Malul

683
6
8

Create json file from dataset in spark scala

1 Answers1