0

Notebook I'm working on for Data Transformation

Problem:

I have this JSON File with employee data that I would like to do some transformations with in Databricks.

I obtained this data set from Bamboo HR API. As you can see in the image there are many tables within this data set. (It should all be in just 1 table)

Anyone got any ideas how to transform it, so I can later copy it in to CSV file in my datalake.

Current Progress

So this is where I am up to now, manage to unpivot 1 of those tables.

However there is many more tables in this data set that needs to be transformed.

I want to transform this JSON file dataset to a CSV file and then the data can be used to upload in to a SQL DB.

Any ideas?

starball
  • 20,030
  • 7
  • 43
  • 238
Hassan
  • 1
  • 2
  • Welcome! Can you please read about [the problems with images of text](//meta.stackoverflow.com/a/285557/11107541) and then [edit] to **convert** your images of text into actual text? Likely useful: [/help/formatting](/help/formatting). See also [/editing-help#tables](/editing-help#tables) for how. You might find [tablesgenerator.com](//www.tablesgenerator.com/markdown_tables) useful. – starball Jan 17 '23 at 23:19
  • Does this answer your question :https://stackoverflow.com/questions/38753898/how-to-flatten-a-struct-in-a-spark-dataframe – Sharma Jan 18 '23 at 08:25

0 Answers0