1

I have a project that generates reports in html string format. Then I attach this string to a label or literal and works great.

The problem is.. when the report is too big, around (27k records), I get the System.OutofMemoryException and this error occurs exactly in the last line: 

Label1.text = totalStringBuilderHtml.ToString()

The program is able to generate around 300k records before I get this exception on the StringBuilder.. and the DOM can renderize up to 216k records using clone() and append() on the first 27k.

So, Is there another way to open this big Html string from server to client in asp.net ?

EDIT: Now it is generating up to 70k records, using a div on a client side, filling it with a JavaScript function and using a ScriptManager to call it.

<asp:ScriptManager ID="ScriptManager1" EnablePageMethods="true" EnablePartialRendering="true" runat="server" />

<div id="divContent"> </div>

Script:

function renderContent(text) {
document.getElementById('divContent').innerHTML = text;
};

Calling the script on code:

myPage.ClientScript.RegisterStartupScript(myPage.GetType(), "rendering", "renderContent(' " + totalStringBuilderHtml.ToString() + " ');", True);

But the DOM can take 3 times more, how can I hit this limit before getting the System.OutOfMemoryException ?

Jeferson Correa
  • 67
  • 6
  • 1
    Why do you need a huge `totalStringBuilderHtml`? Can't you split it and iterate over it in your view? – Patrick Hofman Jan 29 '18 at 14:03
  • 1
    Can you render the page without the content and make an AJAX call to an ASHX handler to fetch only the content and nothing else? That way the ASHX can just stream the string directly, no ASPX page or controls or any of that. – David Jan 29 '18 at 14:04
  • @PatrickHofman I tried to split it into more labels, but that did not make any difference, a label can receive 27k records, but two labels together only accept the half of it (each). – Jeferson Correa Jan 29 '18 at 14:07
  • Does the entire set of records need to be generated *at once* for the report or could you "stream" them to something using an `IEnumerable` and use `yield return` to return each record? That would allow you to use the `IEnumerable<>` as a`DataSource` for a `Repeater` that can render each individual record one at a time. That should significantly cut down on memory. They key part is using `yield` though, so that not everything is in memory at once. – Bradley Uffner Jan 30 '18 at 12:44
  • @JefersonPradoMarques OOM doesn't mean you run out of memory. Most of the time it means the code causes so many reallocations and the memory gets so fragmented that the runtime can't allocate a single block of memory as large as you require. For example, a list grows by *doubling* its internal buffer. If you add 27K items one by one you cause a *lot* of reallocations. – Panagiotis Kanavos Jan 30 '18 at 12:45
  • @JefersonPradoMarques besides, who's going to read the 26990 items in the list that *aren't* visible? Why are you trying to load 27K items in a *label*? What are you trying to do and why do you think creating a bloated page is the answer? – Panagiotis Kanavos Jan 30 '18 at 12:47
  • @JefersonPradoMarques what is the *actual* problem that you want to solve? Whatever it is, creating a huge StringBuilder won't help. Putting 300K items in a div won't help either. *Humans* can't read more than 10-20 items at a time. Are you trying to export data as an HTML file? Display lots of data by paging on the *browser* (forget it)? Or something else? There are other, far easier solutions that could take as little as 5 lines of code – Panagiotis Kanavos Jan 30 '18 at 12:51
  • @PanagiotisKanavos Im creating a Report Generator component for my University, it receives the header, footer, and records in a html string format, and the component is able to measure, add css, and ajust all of content in the pages, respecting the margins, jumping to the next page, etc. After rendering it in a browser it can be saved in a pdf file or printed, that is the intention. Sometimes the report will generate a large amount of data, due to the many students, courses, and diferent combination of informations. The goal here is to increase the amount of the report generated as possible. – Jeferson Correa Jan 30 '18 at 13:23
  • @PanagiotisKanavos The pages of the report are rendered one under the other, and each one attached to a real page when given the command to print or save as a pdf file. – Jeferson Correa Jan 30 '18 at 13:38
  • @JefersonPradoMarques check how reporting engines work then. They *don't* display the entire report at once, they use paging or infinite scrolling. Paging means that you ask the server to send you only the data you are going to display, nothing more. Infinite scrolling works the same way, it's just invisible. You only get the entire file when you save the report. – Panagiotis Kanavos Jan 30 '18 at 13:38
  • @JefersonPradoMarques another option is to write to the Response stream directly, instead of trying to construct the entire string in memory. That's helpful when you want to save a report as a file. No matter how big the resulting file is, the server will use very little memory – Panagiotis Kanavos Jan 30 '18 at 13:41
  • @JefersonPradoMarques and finally, the quick & dirty option is to set the capacity of the StringBuilder to a high-enough value that it won't have to reallocate its buffer. You can calculate the size by multiplying the number of records with the expected string size per record. You'll still end up with slow code that uses a lot of memory but at least you'll avoid the OOM – Panagiotis Kanavos Jan 30 '18 at 13:44
  • @PanagiotisKanavos thanks for all the clarifications, it helped me to understand whats going on.. I will try those things! – Jeferson Correa Jan 30 '18 at 13:48

1 Answers1

0

Create a div that get generated on the server:

<div runat="server" id="myDiv"></div>

In the code behind:

myDiv.InnerHtml = totalStringBuilderHtml.ToString();
Kevin Smith
  • 13,746
  • 4
  • 52
  • 77