0

I have the following dataframe that contains all the data I need. The thing is, each child can be found in the parent column with their own children

  • 1000584 is Top level
  • 4003773 is Level 1
  • 1252665 is Level 2
    *1321212 is Level 3

enter image description here

What I want to achieve is this:

enter image description here

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Laur
  • 71
  • 1
  • 9
  • Check this answer for a similar question https://stackoverflow.com/a/62521978/11713502 – Tushar Patil Jan 06 '23 at 11:05
  • Can you give any sample code that you have tried so far? – Rakesh Govindula Jan 06 '23 at 11:49
  • Yes, I tried to use the self join but seems it's much complex than that. hierarchy = combinedDF.alias('Parrent').join(combinedDF.alias('Child'), col("Parrent.MATNR") == col("Child.IDNRK"), 'left') – Laur Jan 06 '23 at 12:34
  • @TusharPatil That solution was not what I needed. I managed to solve it. I had to create some custom methods in Spark to do the trick. But thanks! – Laur Jan 12 '23 at 09:33

0 Answers0