1

There is a datatype of struct as below:

array<
struct<
          handicap:
            struct<ABC:string,
                   high:string,
                   low:string,
                             
                   >,
                        ,
              index:bigint,
              LGN:bigint,
              LP:
                array
                    <struct
                            <
                            REFNO:string,
                            BNK:boolean,
                            ISA:boolean,
                            isInRunning:boolean,
                            mktref:string,
                            OR:string,
                            PN:bigint,
                            place:bigint,
                    placeTerms:
                            struct
                                <
                                  den:bigint,
                                  num:bigint,
                                  places1:bigint
                                                         
                                >,
                            places1:string,
                            traderUser:string
                            >
                   >
                   >
                   >

How do I capture the path of the actual variable using pyspark. eg: if I want refno: I need the header path as : handicap_LP.ref_no and if I want tofr den: the path I require is handicap.LP.placeterms.den How do I can capture that?

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
Rjuser
  • 21
  • 5
  • 1
    does this answer your question? https://stackoverflow.com/questions/61618715/is-there-a-way-to-collect-the-names-of-all-fields-in-a-nested-schema-in-pyspark – AdibP Jul 12 '21 at 11:02
  • Hi @AdibP, thanks for pointing it to the right link.. wondering if there is similar process to get the values as well from this column? currently I am looping through the structure and exploding this way I will not be able to map the column names to column values. is there any other way I can get the values as well in the order? – Rjuser Jul 12 '21 at 12:19

0 Answers0