1

There is a table which has 10 columns. One of the column type is struct And suppose that column name is xyz. And xyz contains array of elements.

e.g., xyz column contains [a : value 1, b: value 2, c:value 3...] Now I want to remove c element from this array.

How do I do it?

This code needs to be done in pyspark/python.

shivani
  • 11
  • 4
  • 2
    Please give an example code and in this case structure of your data for others to help. – xtlc Sep 22 '21 at 09:38
  • You can check here - https://stackoverflow.com/questions/57758729/how-to-dynamically-slice-an-array-column-in-spark – Vaebhav Sep 22 '21 at 10:09

1 Answers1

0

You must flatten your struct using something like df.select('xyz.*'), then restructure it using array function, struct function as needed.

pltc
  • 5,836
  • 1
  • 13
  • 31