Flatten a String Datatype of XML Content in Pyspark

Asked Apr 14 '22 at 02:36

Active Apr 14 '22 at 02:36

Viewed 120 times

I need to convert the XML_TEST to XML readable schema so that it can be flattened based on Array Data Type and achieve the desired output provided below.

Input Data: DataFrame:

ID|NAME|XML_TEXT

1|XXXX|Nested XML

2|YYYY|nested XML

Schema:

ID String

Name String

XML_TEST String

output:

ID|NAME|XML_ID1|COL2|COL3

1|XXXX|10|ABC|DEF

2|YYYY|11|GGGG|DDD

If it was a direct XML we can achieve this using root and row tags but here(XML_TEXT) is of String Data Type and also need to be flattened.

asked Apr 14 '22 at 02:36

user18798951

Does this answer your question? [Read XML in spark](https://stackoverflow.com/questions/50429315/read-xml-in-spark) – teedak8s Apr 14 '22 at 02:47
No, my xmlcontent is in data frame column and of string data type. – user18798951 Apr 14 '22 at 11:16
https://stackoverflow.com/a/65005049/2986344 Sorry for the wrong link. – teedak8s Apr 15 '22 at 01:43

Flatten a String Datatype of XML Content in Pyspark

0 Answers0