Deciding on when to use XmlDocument vs XmlReader

Question

I'm optimizing a custom object -> XML serialization utility, and it's all done and working and that's not the issue.

It worked by loading a file into an XmlDocument object, then recursively going through all the child nodes.

I figured that perhaps using XmlReader instead of having XmlDocument loading/parsing the entire thing would be faster, so I implemented that version as well.

The algorithms are exactly the same, I use a wrapper class to abstract the functionality of dealing with an XmlNode vs. an XmlReader. For instance, the GetChildren methods yield returns either a child XmlNode or a SubTree XmlReader.

So I wrote a test driver to test both versions, and using a non-trivial data set (a 900kb XML file with around 1,350 elements).

However, using JetBrains dotTRACE, I see that the XmlReader version is actually slower than the XmlDocument version! It seems that there is some significant processing involved in XmlReader read calls when I'm iterating over child nodes.

So I say all that to ask this:

What are the advantages/disadvantages of XmlDocument and XmlReader, and in what circumstances should you use either?

My guess is that there is a file size threshold at which XmlReader becomes more economical in performance, as well as less memory-intensive. However, that threshold seems to be above 1MB.

I'm calling ReadSubTree every time to process child nodes:

public override IEnumerable<IXmlSourceProvider> GetChildren ()
{
    XmlReader xr = myXmlSource.ReadSubtree ();
    // skip past the current element
    xr.Read ();

    while (xr.Read ())
    {
        if (xr.NodeType != XmlNodeType.Element) continue;
        yield return new XmlReaderXmlSourceProvider (xr);
    }
}

That test applies to a lot of objects at a single level (i.e. wide & shallow) - but I wonder how well XmlReader fares when the XML is deep & wide? I.e. the XML I'm dealing with is much like a data object model, 1 parent object to many child objects, etc: 1..M..M..M

I also don't know beforehand the structure of the XML I'm parsing, so I can't optimize for it.

I've always wondered why there was both an XmlDocument and an XmlReader... — James Jones, Oct 01 '09 at 16:41
Actually there is another option to XMLDocument and XMLReader. You can now use LINQ to XML but actually XMLReader is more efficient in most ways. — Tarik, Oct 01 '09 at 16:51
Wait. Your `GetChildren` method returns an `XmlReader`? You mean, you're calling `XmlReader.Create()` every time you process a child node? — Robert Rossney, Oct 01 '09 at 18:48
Most code using `XmlReader` doesn't use `ReadSubtree`, so that's a bad comparison. Also, you need a `using` block for `xr`. — John Saunders, Oct 01 '09 at 20:38
Make XML document as NULL after use or use xml document inside using Blocks. — Banketeshvar Narayan, Jan 13 '14 at 08:55

score 76 · Accepted Answer · edited Jun 16 '13 at 01:16

I've generally looked at it not from a fastest perspective, but rather from a memory utilization perspective. All of the implementations have been fast enough for the usage scenarios I've used them in (typical enterprise integration).

However, where I've fallen down, and sometimes spectacularly, is not taking into account the general size of the XML I'm working with. If you think about it up front you can save yourself some grief.

XML tends to bloat when loaded into memory, at least with a DOM reader like XmlDocument or XPathDocument. Something like 10:1? The exact amount is hard to quantify, but if it's 1MB on disk it will be 10MB in memory, or more, for example.

A process using any reader that loads the whole document into memory in its entirety (XmlDocument/XPathDocument) can suffer from large object heap fragmentation, which can ultimately lead to OutOfMemoryExceptions (even with available memory) resulting in an unavailable service/process.

Since objects that are greater than 85K in size end up on the large object heap, and you've got a 10:1 size explosion with a DOM reader, you can see it doesn't take much before your XML documents are being allocated from the large object heap.

XmlDocument is very easy to use. Its only real drawback is that it loads the whole XML document into memory to process. Its seductively simple to use.

XmlReader is a stream based reader so will keep your process memory utilization generally flatter but is more difficult to use.

XPathDocument tends to be a faster, read-only version of XmlDocument, but still suffers from memory 'bloat'.

Loading XML documents, however large, into memory does NOT cause large objects. Holding the XML as a string however does! It is the size of the individual objects that matter with respect to the GCs ability to defragment memory, but the total size of the object graph that matters with respect to memory usage. — The Dag, Mar 27 '12 at 10:57
FWIW I just did a benchmark between XDocument, XMLReader, and XmlDocument. To do similar paths they took 0.004, 0.001, and 0.692 seconds respectively. — micahhoover, Jul 11 '14 at 19:16

score 11 · Answer 2 · answered Oct 01 '09 at 20:08

11

XmlDocument is an in-memory representation of the entire XML document. Therefore if your document is large, then it will consume much more memory than if you had read it using XmlReader.

This is assuming that when you use XmlReader you read and process the elements one-by-one then discard it. If you use XmlReader and construct another intermediary structure in memory then you have the same problem, and you're defeating the purpose of it.

Google for "SAX versus DOM" to read more about the difference between the two models of processing XML.

answered Oct 01 '09 at 20:08

dso

9,463
10
53
59

1

The annoying thing is there is absolutely no indication at all where (ballpark) a document becomes "large" and XmlReader starts to yield any considerable size benefit. Is it 1KB, 1MB, or even much more? I'm sure the answer is "it depends", but without a clue at all we're left to determine these things experimentally on a case-by-case basis, except in cases where being able to handle arbitrarily large data is a requirement (then XmlReader is the clear choice). – The Dag Mar 27 '12 at 11:04

score 4 · Answer 3 · answered Apr 13 '15 at 16:03

Another consideration is that XMLReader might be more robust for handling less-than-perfectly-formed XML. I recently created a client which consumed an XML stream, but the stream didn't have the special characters escaped correctly in URIs contained in some of the elements. XMLDocument and XPathDocument refused to load the XML at all, whereas using XMLReader I was able to extract the information I needed from the stream.

score 0 · Answer 4 · answered Nov 11 '12 at 15:52

The encoding difference is because two different measurements are being mixed. UTF-32 requires 4 bytes per character, and is inherently slower than single byte data.

If you look at the large (100K) element test, you see that the time increasesw by about 70mS for each case regardless of the loading method used.

This is a (nearly) constant difference caused specifically by the per character overhead,

score 0 · Answer 5 · answered Oct 01 '09 at 16:43

There is a size threshold at which XmlDocument becomes slower, and eventually unusable. But the actual value of the threshold will depend on your application and XML content, so there are no hard and fast rules.

If your XML file can contain large lists (say tens of thousands of elements), you should definitely be using XmlReader.

Deciding on when to use XmlDocument vs XmlReader

5 Answers5

Linked

Related