know a better way to read big xml files in c#

Question

Hi guys I want to know a better way to read xml files in c#. I have a 4.5 GB file and i am using the StreamReader class. But it's too slow.

Here is a part of my code:

        using (StreamReader s = new StreamReader(NmArquivo))
        StreamReader s = new StreamReader(NmArquivo);
        {
            XDocument xmlFile = XDocument.Load(s);

            IEnumerable<XElement> regsXml = from el in xmlFile.Descendants("ContainedResourceList").Elements(tpObra)
                                            select el;

            seqObra = 0;
            statusProcesso.Value = 0;
            statusProcesso.Maximum = regsXml.Count();
            Application.DoEvents();

            foreach (XElement regXml in regsXml)
            {

                DDExObra obra = new DDExObra();
                obra.Controle = controleDDEx.controle;
                obra.UsuarioInclusao = controleDDEx.UsuarioInclusao;
                obra.SequencialObra = seqObra;
                obra.ResourceReference = string.IsNullOrEmpty((string)regXml.Element("ResourceReference")) ? null : regXml.Element("ResourceReference").Value;
                obra.Title = string.IsNullOrEmpty((string)regXml.Element("ReferenceTitle")) ? null : regXml.Element("ReferenceTitle").Value;
                obra.Duration = string.IsNullOrEmpty((string)regXml.Element("Duration")) ? null : regXml.Element("Duration").Value;
                obra.DsDuration = string.IsNullOrEmpty(obra.Duration) ? null : XmlConvert.ToTimeSpan(regXml.Element("Duration").Value).ToString();
                seqObra++;

                // identificadores
                var q1 = from el in regXml.Elements(tpIdObra)
                         select new
                         {
                             ISRC = string.IsNullOrEmpty((string)el.Element("ISRC")) ? null : el.Element("ISRC").Value,
                             ProprietaryId = string.IsNullOrEmpty((string)el.Element("ProprietaryId")) ? null : el.Element("ProprietaryId").Value,
                             CatalogNumber = string.IsNullOrEmpty((string)el.Element("CatalogNumber")) ? null : el.Element("CatalogNumber").Value
                         };

                var r1 = q1.FirstOrDefault();
                if (r1 != null)
                {
                    obra.ProprietaryId = r1.ProprietaryId;
                    obra.Isrc = r1.ISRC;
                    obra.CatalogNumber = r1.CatalogNumber;
                }
                r1 = null;
                q1 = null;

Is there a better way ?

What is slow? Have you profiled the code to determine the exact part of the process that is taking too long? — John Koerner, Jul 22 '14 at 14:58
my advice would be to break your file down into smaller files, or use a sql DB instead — Sam I am says Reinstate Monica, Jul 22 '14 at 15:00
If you need optimization, this code is better suited to CodeReview.SE and not StackOverflow. — WGS, Jul 22 '14 at 15:06

score 1 · Answer 1 · edited May 23 '17 at 12:13

XmlReader is the appropriate class to use if you are just reading Xml from beginning to end. XDocument and XmlDocument are appropriate for keeping the entire document in memory. What is the best way to parse (big) XML in C# Code?

However: To @Joel Koerner's point, you must first prove that the XDocument is the bottleneck here. Although it is very possible, since I've never had to manage an Xml file that large before.

know a better way to read big xml files in c#

1 Answers1