Provide XML file for download with large number of records in asp.net c#

Question

I need to provide xml file for download around with 6,00,000 records(data may increase) but not allowed to save file on disk. I was facing issue in directly writing xml to stream & then providing for download , so i have first created xml file on disk & writing data to it and trying to read it in byes & provide for download and then delete file from disk. But getting "system.outofmemoryexception" in "Byte[] b = File.ReadAllBytes(filepath);".

Below is my code:

string name = string.Format("{0:yyyyMMddHHmmss}", DateTime.Now);
string filename = "TestFile.xml";

string filepath = ConfigurationManager.AppSettings["XmlFiles"] + "\\" + filename;


DataTable dataTable = dsData.Tables[0];             

FileStream fs =new FileStream(filepath, FileMode.Create);
XmlWriterSettings xws = new XmlWriterSettings { OmitXmlDeclaration = true };
using (XmlWriter xmlWriter = XmlWriter.Create(fs,xws))
{

  xmlWriter.WriteStartElement("root");

  foreach (DataRow dataRow in dataTable.Rows)
  {
    xmlWriter.WriteStartElement("datanode");
    foreach (DataColumn dataColumn in dataTable.Columns)
    {
     xmlWriter.WriteElementString(dataColumn.ColumnName.Replace("\n\r", " ")
                                  .Replace("\n", " ").Replace("\r", " "), Convert.ToString(dataRow[dataColumn]).Replace("\n\r", " ").Replace("\n", " ").Replace("\r", " "));
                        }

         xmlWriter.WriteEndElement();
     }
     xmlWriter.WriteEndElement();
     xmlWriter.Flush();
     xmlWriter.Close();
   }

   fs.Close();
   Byte[] b = File.ReadAllBytes(filepath);                
   if (File.Exists(filepath))
      File.Delete(filepath);

string s = string.Format("{0:yyyyMMddHHmmss}", DateTime.Now);
HttpContext.Current.Response.ContentType = "application/xml";
HttpContext.Current.Response.ContentEncoding = System.Text.Encoding.UTF8;
HttpContext.Current.Response.AppendHeader("Content-Disposition", "attachment; filename=" + s + ".xml" + "");
HttpContext.Current.Response.BinaryWrite(b);
HttpContext.Current.Response.End();

Is there any other way to handle large number of records?

Perhaps [this](http://stackoverflow.com/questions/2809514/outofmemoryexception-when-i-read-filestream-500-mb) thread will help you. — Tobias Johansson, Jul 11 '14 at 06:32

rene · Accepted Answer · 2014-07-14T08:25:05.077

You can directly hookup the XmlWriter to the Response OutputStream without the need for an intermediate file. By setting BufferOutput to false you prevent that the complete repsonse is buffered server side before being send to the client.

Do notice that the multiple Replace statements will lead to extra memory pressure/more strings to be GC-ed.

string name = string.Format("attachment; filename={0:yyyyMMddHHmmss}.xml", DateTime.Now);
HttpContext.Current.Response.ContentType = "application/xml";
HttpContext.Current.Response.ContentEncoding = System.Text.Encoding.UTF8;

HttpContext.Current.Response.BufferOutput = false; //start streaming immideately

HttpContext.Current.Response.AppendHeader("Content-Disposition", name);

DataTable dataTable = dsData.Tables[0];             

XmlWriterSettings xws = new XmlWriterSettings { OmitXmlDeclaration = true };

using (XmlWriter xmlWriter = XmlWriter.Create(
         HttpContext.Current.Response.OutputStream,  // The OutputStream of the HttpResponse
         xws))
{
  xmlWriter.WriteStartElement("root");

  var cleanedColNames = new Dictionary<DataColumn, string>();
  foreach (DataColumn dataColumn in dataTable.Columns)
  {
      cleanedColNames.Add(dataColumn, 
          dataColumn.ColumnName.Replace("\n\r", " ")
                               .Replace("\n", " ")
                               .Replace("\r", " "));
  }

  foreach (DataRow dataRow in dataTable.Rows)
  {
    xmlWriter.WriteStartElement("datanode");
    foreach (DataColumn dataColumn in dataTable.Columns)
    {
         xmlWriter.WriteElementString(
             cleanedColNames[dataColumn],  
             Convert.ToString(dataRow[dataColumn]).Replace("\n\r", " ")
                                                  .Replace("\n", " ")
                                                  .Replace("\r", " "));
                        }
         xmlWriter.WriteEndElement();
     }
     xmlWriter.WriteEndElement();
     xmlWriter.Flush();
     xmlWriter.Close();
   }

HttpContext.Current.Response.End();

if file is bigger then would it be ok to use Response OutputStream? i have doubt that it will occupy more cache. — GMD, Jul 14 '14 at 03:52
I added an extra setting for BufferOutput to prevent serverside buffering. I'm pretty sure my solution uses less memory than yours. Do note that your Replace statements cause memory pressure as well. — rene, Jul 14 '14 at 08:16
It worked, and for replace statement i will need to have it for some string columns, for rest i will remove it. — GMD, Jul 14 '14 at 10:30

Provide XML file for download with large number of records in asp.net c#

1 Answers1