Convert pyspark dataframe to json file

Asked Aug 18 '21 at 19:00

Active Aug 18 '21 at 20:18

Viewed 96 times

I have a dataframe below and want to write that contents to a .json file.

Sample input dataframe and expected output json

And while creating output files , I do not want success part log files, so I tried to collect () the values from dataframe and used json_dumps() to create the file. But i am losing the column names and formats as opposed to the expected format in picture

Please help!

edited Aug 18 '21 at 20:18

werner

13,518
6
30
45

asked Aug 18 '21 at 19:00

Dinesh Kumar Rajendran

2

Please post the code you tried, not just a picture. – Captain Jack Sparrow Aug 18 '21 at 19:33
Does [this](https://stackoverflow.com/questions/43269244/pyspark-dataframe-write-to-single-json-file-with-specific-name) answer your question? – Ofek Glick Aug 21 '21 at 07:06

Convert pyspark dataframe to json file

0 Answers0