how to parse unstructured xml file using python?

Question

How can i parse unstructured xml file? i need to get data inside patient tag and title using elementTree.

<?xml version="1.0" encoding="UTF-8"?>
<ClinicalDocument xmlns="urn:hl7-org:v3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hl7-org:v3 CDA.xsd">
<templateId root="2.16.840.1.113883.10.20.22.1.1"/>
<id extension="4b78219a-1d02-4e7c-9870-dc7ce3b8a8fb" root="1.2.840.113619.21.1.3214775361124994304.5.1"/>
<code code="34133-9" codeSystem="2.16.840.1.113883.6.1" codeSystemName="LOINC" displayName="Summarization of episode note"/>
<title>Summary</title>
<effectiveTime value="20170919160921ddfdsdsdsd31-0400"/>
<confidentialityCode code="N" codeSystem="2.16.840.dwdwddsd1.113883.5.25"/>
<recordTarget>
<patientRole><id extension="0" root="1.2.840.113619.21.1.3214775361124994304.2.1.1.2"/>
<addr use="HP"><streetAddressLine>addd2 </streetAddressLine><city>fgfgrtt</city><state>tr</state><postalCode>121213434</postalCode><country>rere</country></addr>
<patient>
<name><given>fname</given><family>lname</family></name>
<administrativeGenderCode code="F" codeSystem="2.16.840.1.113883.5.1" displayName="Female"/>
<birthTime value="19501025"/>
<maritalStatusCode code="M" codeSystem="2434.16.840.1.143434313883.5.2" displayName="M"/>
<languageCommunication>
<languageCode code="eng"/>
<proficiencyLevelCode nullFlavor="NI"/>
<preferenceInd value="true"/>
</languageCommunication>
</patient>

i want given name , family name , gender and title.

Beautifull soup can handle bad xml https://www.crummy.com/software/BeautifulSoup/bs4/doc/ — mbieren, Jun 25 '19 at 13:28

score 2 · Accepted Answer · edited Jun 04 '21 at 16:11

Using BeautifulSoup bs4 and lxml parser library to scrape xml data.

from bs4 import BeautifulSoup

xml_data = '''<?xml version="1.0" encoding="UTF-8"?>
<ClinicalDocument xmlns="urn:hl7-org:v3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hl7-org:v3 CDA.xsd">
<templateId root="2.16.840.1.113883.10.20.22.1.1"/>
<id extension="4b78219a-1d02-4e7c-9870-dc7ce3b8a8fb" root="1.2.840.113619.21.1.3214775361124994304.5.1"/>
<code code="34133-9" codeSystem="2.16.840.1.113883.6.1" codeSystemName="LOINC" displayName="Summarization of episode note"/>
<title>Summary</title>
<effectiveTime value="20170919160921ddfdsdsdsd31-0400"/>
<confidentialityCode code="N" codeSystem="2.16.840.dwdwddsd1.113883.5.25"/>
<recordTarget>
<patientRole><id extension="0" root="1.2.840.113619.21.1.3214775361124994304.2.1.1.2"/>
<addr use="HP"><streetAddressLine>addd2 </streetAddressLine><city>fgfgrtt</city><state>tr</state><postalCode>121213434</postalCode><country>rere</country></addr>
<patient>
<name><given>fname</given><family>lname</family></name>
<administrativeGenderCode code="F" codeSystem="2.16.840.1.113883.5.1" displayName="Female"/>
<birthTime value="19501025"/>
<maritalStatusCode code="M" codeSystem="2434.16.840.1.143434313883.5.2" displayName="M"/>
<languageCommunication>
<languageCode code="eng"/>
<proficiencyLevelCode nullFlavor="NI"/>
<preferenceInd value="true"/>
</languageCommunication>
</patient>'''


soup = BeautifulSoup(xml_data, "lxml")

title = soup.find("title")
print(title.text.strip())

patient = soup.find("patient")
given = patient.find("given").text.strip()
family = patient.find("family").text.strip()
gender = patient.find("administrativegendercode")['displayname'].strip()

print(given)
print(family)
print(gender)

O/P:

Summary
fname
lname
Female

Install library dependency:

pip3 install beautifulsoup4==4.7.1
pip3 install lxml==4.3.3

what is the best way to read multiple xml files from folder using python? — prasanna, Jun 25 '19 at 13:54
@prasanna try this https://stackoverflow.com/questions/52247468/open-and-read-multiple-xml-files-from-the-folder-python — bharatk, Jun 25 '19 at 14:06
it is working fine, Thank You. but this python program is expecting both file and xml file must be in same folder?? — prasanna, Jun 25 '19 at 14:18
i am getting this error 'OSError: [Errno 2] No such file or directory: './user/root/SQP/raw_ccd/*'' in linux. — prasanna, Jun 25 '19 at 14:33
@prasanna Please ask a new question, what you have tried python code with an error message. — bharatk, Jun 26 '19 at 05:22
above parsing can i do using elementTree? facing some issue with BeautifulSoup — prasanna, Jun 26 '19 at 12:55

score 1 · Answer 2 · answered Jun 25 '19 at 13:58

1

Or you can simply use lxml. Here is tutorial that I used: https://lxml.de/tutorial.html But it should be similar to:

from lxml import etree
root = etree.Element("patient")
print(root.find("given"))
print(root.find("family"))
print(root.find("give"))

answered Jun 25 '19 at 13:58

kkarczewski

331
3
19

all the values are printing none. – prasanna Jun 26 '19 at 13:39
all values are printing none. please suggest me. – prasanna Jun 26 '19 at 15:24
Have you tried xpath with lxml? Here you have link to my github: https://github.com/kkarczewski/xml_operation/blob/master/xml_modify/xml_operation.py Try with first function `get_element(path, xpath)`. – kkarczewski Jun 27 '19 at 07:25
you can modify that function to read text from string. Instead of `tree = etree.parse(path) root = tree.getroot()` you can use `root = etree.fromstring(xml)` – kkarczewski Jun 27 '19 at 09:23
still i am getting None only. – prasanna Jun 27 '19 at 10:47
etree.parse(path) is printing complete xml file multiple times (centos or linux). – prasanna Jun 27 '19 at 14:03

how to parse unstructured xml file using python?

2 Answers2