Using GraphFrames (Scala) to compute hierarchy

Question

I have a dataframe below:

employee_id|employee_name|manager_employee_id|
----------------------------------------------
    1           eric (ceo)      1
    2           edward          1
    3           john            1
    4           james           2
    5           ella            4

I would like to use spark (scala) graphframe logic to achieve the following:

employee_id|employee_name|manager_employee_id|level|hierarchy
-------------------------------------------------------------
    1           eric            1                0    /1
    2           edward          1                1    /1/2
    3           john            1                1    /1/3
    4           james           2                2    /1/2/4
    5           ella            3                3    /1/2/4/5

Any help would be much appreciated

score 0 · Answer 1 · edited Nov 12 '20 at 12:59

0

I think the answer you are looking for is more or less related to this.

The only modification required at your end will be aggregating the messages to get the complete hierarchy . For that part you can refer to this.

A combination of above two will get you the desired results.

edited Nov 12 '20 at 12:59

tomerpacific

4,704
13
34
52

answered Nov 12 '20 at 10:50

ironhide96

11
2

Using GraphFrames (Scala) to compute hierarchy

1 Answers1