Why does this XML file load slowly?

Question

I have some very simple code:

        XmlDocument doc = new XmlDocument();
        Console.WriteLine("loading");
        doc.Load(url);
        Console.WriteLine("loaded");

        XmlNodeList nodeList = doc.GetElementsByTagName("p");

        foreach(XmlNode node in nodeList)
        {
            Console.WriteLine(node.ChildNodes[0].Value);
        }
        return source;

I'm working on this file and it takes two minutes to load. Why does it take so long? I tried both with fetching and file from the net and loading a local file.

Uh your link to the file is to a wikipedia article for "hot milk cake" — Daniel DiPaolo, Apr 14 '11 at 00:49
You maybe want to check that url again, pretty sure C#'s XML library doesn't load hot milk cakes. — Juliet, Apr 14 '11 at 00:50
Hot Milk Cake does not pass XML validator test on http://www.w3schools.com/xml/xml_validator.asp. One can argue that the .Net doesn't have to load it at all. — seva titov, Apr 14 '11 at 01:09

score 9 · Accepted Answer · edited May 23 '17 at 11:59

9

I imagine it's the DTD of the page that's taking so long to load. Given that it defines entities, you shouldn't disable it, so you're probably better off not going down this path.

Given the inner workings of the wikipedia parser (a right mess), I'd say it's a big leap to assume it's going to produce well-formed XHTML every time.

Use HTML Agility Pack to parse (then you can convert to XmlDocument a little more easily if required, IIRC).

If you really want to go down the XmlDocument route you can keep a local cache of the HTML DTDs. See this post, this post and this post for details.

edited May 23 '17 at 11:59

Community

1
1

answered Apr 14 '11 at 00:54

spender

117,338
33
229
351

+1, beat me to the answer; if you download a copy and remove the DTD it parses right away, but then fails because ® is only defined in the DTD. – meklarian Apr 14 '11 at 01:00
3

The W3C throttles traffic to their DTD files, because they get buried in requests. You could use a custom entityresolver to load local copies of the DTD files. – Mads Hansen Apr 14 '11 at 01:26

score 5 · Answer 2 · answered Apr 14 '11 at 00:59

It is becuase XmlDocument doesn't just load your Xml into a nice class heirarchy it also goes and fetches all of the namespace DTD's defined in the document. Run fiddler and you will see the calls to fetch

http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent
http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent
http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent

These all took me about 20 seconds to fetch.

Why does this XML file load slowly?

2 Answers2