Fixing xml.etree.ElementTree.ParseError: undefined entity è

Question

I was working with a .pri file. Which has a xml format. Like below.

<?xml version="1.0"?>
<!DOCTYPE text SYSTEM "text.dtd">
<text id="fn000001">
  <au id="fn000001.1" s="N00023">
    <w id="fn000001.1.1">                          hi              </w>
    <w id="fn000001.1.2">                          there           </w>
    <l id="fn000001.1.3">                          ?               </l>
  </au>
</text>

Now if I call a single file, by using below command, it works properly.

import xml.etree.ElementTree as ET
tree = ET.parse('/path/fn000001.pri')
root = tree.getroot()
print(root.get('id'))

Now I want to apply this to all the .pri files in the folder. For that, I am using below command,

import glob
import xml.etree.ElementTree as ET
a = glob.glob('/path/*.pri')
  
for files in a:
    tree = ET.parse(files)
    print(tree)

That throws the error,

tree = ET.parse(files)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 1196, in parse
    tree.parse(source, parser)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 597, in parse
    self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: undefined entity &egrave;: line 147, column 52

Please suggest possible solutions. The related .dtd is in the same folder.

ElementTree does not support entities declared in a separate DTD file. See https://stackoverflow.com/q/60746298/407651 — mzjn, Jun 15 '21 at 11:47
Yes, I referred to that question. In the question, it is suggested to add `<!ENTITY Ouml 'Ö'>` But in my .dtd file, `<!ENTITY egrave "è">` is already been added. Still getting error, @mzjn — adikh, Jun 15 '21 at 11:51
As I said, ElementTree does not support entities declared in a separate DTD file. Either declare entities in the XML file or use lxml. Or don't use entities at all. — mzjn, Jun 15 '21 at 11:54
Does this answer your question? [ParseError: undefined entity while parsing XML file in Python](https://stackoverflow.com/questions/60746298/parseerror-undefined-entity-while-parsing-xml-file-in-python) — Joe, Jun 15 '21 at 11:55
@Joe No! In that question, the user works with a single file. While working with multiple files from a folder I am getting this error. With a single XML file I am getting a desired output. Now if " Either declare entities in the XML file" then, in which XML file from multiple XML files available. I am trying with lxml. — adikh, Jun 15 '21 at 12:00

adikh · Accepted Answer · 2021-06-16T03:13:03.170

Ok! I came across many posts and answers related to this question. As per the reasons behind this error provided in one comment,

As I said, ElementTree does not support entities declared in a separate DTD file. Either declare entities in the XML file or use lxml. Or don't use entities at all.

So, main question is what to do so that, ElementTree supports the entities declared in a separate DTD file.

Solution 1 : Declare entities in the XML file or use lxml So I used lxml but did not declare the entities in the XML file. It will not solve your problem. And if you declare the entities in XML file, then it does not matter if you are using lxml package or not. So, declare the entities in the XML file as below.

 <!DOCTYPE text [
 <!ENTITY egrave "&#232;">
 ]>
 <text id="fn000001">
   <au id="fn000001.1" s="N00023">
     <w id="fn000001.1.1">                          hi              </w>
     <w id="fn000001.1.2">                          there           </w>
     <l id="fn000001.1.3">                          ?               </l>
   </au>
 </text>

This solution is provided here. ParseError: undefined entity while parsing XML file in Python

But what if you have 1000s of XML files and you want to parse them all at the same time? Then this solution will not work out.

Solution 2: This one is a permanent one. You need to map your dtd file with your .XML file. This is just one line of code. I used lxml package. Check the below code snippet.

    from lxml import etree
    parser = etree.XMLParser(dtd_validation=True)
    tree = etree.parse("file.xml", parser)

All you need to give dtd_validation=True and the code will fetch the information from .dtd file and map it with your .XML file. Make sure .dtd file is in the same directory of your all XML files.

score 0 · Answer 2 · answered May 15 '23 at 22:01

The egrave looks like a standard html5 entity. It may help to convert those to their unicode characters before running the xml parser. In python3.4+ you can use html.unescape for that.

from html import escape, unescape
textXML = re.sub("\\&\\w+\\;", lambda x: escape(unescape(x.group(0))), text)

Fixing xml.etree.ElementTree.ParseError: undefined entity è

2 Answers2