C# Speed up large XML generation

Question

I am writing a C# utility to generate a large XML SOAP request to send a list of products and their attributes to a third party application, which expects the XML in a specific way. Each product has about 30 XML tags. It is not possible to change the XML schema, or convert to JSON.

I am processing around 3000 products, which is roughly taking 25 seconds on my test machine. The products will increase manifold times in future.

Is there a way to speed up the XML generation process? I am primarily using XmlDocument's CreateElement and CreateAttribute (for nodes with attributes), inside a loop, and then covert the final document to string to send as an HTTP request to a locally deployed third party app.

Thanks.

You can use Linq To XML. 25 seconds for 3000*30 tags is too much. — Cetin Basoz, Nov 07 '16 at 22:37
I tested by persisting the XML to a file. The file size comes to about 4 MB. I am testing this on Core i3, Win 7 64-bit OS, 4 GB RAM. — AllSolutions, Nov 07 '16 at 23:25
Without any idea of your existing code, there's not much chance for us to answer. Are you even using `XmlSerializer`? `DataContractSerializer`? Or are you really doing handcrafted serialization with `XmlDocument`? If so, why? And have you tried [profiling](https://stackoverflow.com/questions/3927/what-are-some-good-net-profilers) your existing implementation? — dbc, Nov 07 '16 at 23:36
I am not doing any serialization. The file was just to test the size of the XML. I am directly sending the XML over an HTTP request to the third party app. — AllSolutions, Nov 07 '16 at 23:41
I will probably try to post a trimmed down version of the code sometime later. In the mean time, any tips, like the one suggesting the use of XmlWriter instead of XmlDocument are welcome. — AllSolutions, Nov 07 '16 at 23:42

score 1 · Answer 1 · answered Nov 08 '16 at 13:02

I speed is your main concern, don't use an XML DOM as provided by the XmlDocument class, use an XmlWriter instead:

using System;
using System.Text;
using System.Xml;

class Program
{
    static void Main(string[] args)
    {
        XmlWriter xmlWriter = XmlWriter.Create("foo.xml");

        xmlWriter.WriteStartDocument();
        xmlWriter.WriteStartElement("root");

        xmlWriter.WriteStartElement("someElement");
        xmlWriter.WriteAttributeString("anAttribute", "42");
        xmlWriter.WriteString("Node Content");
        xmlWriter.WriteEndElement();

        xmlWriter.WriteStartElement("someElement");
        xmlWriter.WriteAttributeString("anAttribute", "39");
        xmlWriter.WriteString("Node Content");

        xmlWriter.WriteEndDocument();
        xmlWriter.Close();
    }
}

Swetha · Answer 2 · 2016-11-08T12:54:57.153

0

If you are creating the Xml then try going for XmlWriter as it uses less memory.

edited Nov 08 '16 at 12:54

answered Nov 07 '16 at 22:49

Swetha

457
4
7

Peter - Reinstate Monica · Answer 3 · 2016-11-08T13:13:27.873

The MSDN has the following comment:

XmlDocument. The XmlDocument class is an in-memory or cached tree representation of an XML document. You can use it to edit and navigate the document. While XmlDocument is easy to use, it is very resource-intensive. You should generally use XmlReader and XmlWriter for better performance.

Later the author seems to be concerned primarily about the memory consumption which should only lead to performance issues if memory must be swapped to disk. 30k items don't seem enough to exhaust 4 GB RAM, but it depends on other resource hogs (is a database running?). The author seems to suggest that the XmlDocument class is for small data sets; large amounts of data are better handled by the reader/writer classes, possibly in combination with using XmlDocument for editing small portions of it.

The article generally seems a good read for your issue.

C# Speed up large XML generation

3 Answers3