pySpark Transpose dataframe from a json file and add a header;

Asked Dec 17 '19 at 19:47

Active Dec 17 '19 at 19:47

Viewed 186 times

I have a dataframe as given below. The dataframe is imported from a .json file; and I'm required to use pyspark. The Column headers are also unknown.

A|B|C|D|E|F|...
1|2|3|4|5|6|...

I need help on transposing the above dataset, and output should be displayed as below. And add a header as well

Header 1   Header 2
   A     |    1     |
   B     |    2     |
   C     |    3     |
   D     |    4     |
   E     |    5     |
   F     |    6     |
   .     |    .     |
   .     |    .     |

asked Dec 17 '19 at 19:47

John Boe

3

Start with this: https://stackoverflow.com/questions/37864222/transpose-column-to-row-with-spark – blackbishop Dec 17 '19 at 21:11
This might help : https://community.cloudera.com/t5/Support-Questions/How-to-transpose-a-pyspark-dataframe/td-p/131471 – Ajinkya Bhore Dec 18 '19 at 05:27

pySpark Transpose dataframe from a json file and add a header;

0 Answers0