I have a dataframe with a fixed size array column. Like this:
[v1, v2, v3, v4]
I need to convert the array to a json of the following structure:
{
v1: {
Min: v2,
Max: v3,
Count: v4
}
}
While it is easy to achieve to inner structure, having v1
as the name of the property is more challenging.
I tried 'to_json'
but the keys are taken from column names. In my case 'v1'
changes in each row.
Is it possible to achieve this in pyspark without using a udf? If it helps, I am running on top of Databricks.