Why StringBuilder can't hold huge strings and throwing OOM Exception

Question

I am converting a HTML string to PDF by using iTextSharp XMLWorker. previuosly this was working fine and generates nice pdf files but later I added base64 string Images to My HTML string which I want to show in my PDF. it is working fine for few images( small string ) but when number of images increases parallelly the size of the string increases. I am returning HTML string from a function like this

sb = new StringBuilder();
tw = new StringWriter(sb);
hw = new HtmlTextWriter(tw);
tblDistrictwiseResults.RenderControl(hw);
return sb.ToString();

and now returned HTML string is appending to another string like below

StringBuilder sb = new StringBuilder();

sbFooter = new StringBuilder();
tw = new StringWriter(sbFooter);
hw = new HtmlTextWriter(tw);
tblFooter.RenderControl(hw);


string pdfString = GetPDFString(paperCode, paperName, true);  //This is the string returned from anothe function
sb.Append(pdfString + "<br />" + sbFooter.ToString()); //concatinating returned string and footer string and saving in another stringbuilder( here I am gettiong OOM Exception )

and the string becomes very large but it did not exceed maxcapacity of the StringBuilder.

Please someone help me

EDIT: I am using VS2010(32 bit) and DEV Server 32 bit

"previuosly this was working fine and generates nice pdf files but later I added base64 string Images to My HTML string which I want to show in my PDF." It does not realy mater what your memory is. **Images** that you then **Base64** encode are going to break it. | You should try not to make the whole document in one go. Maybe chapter by chapter? — Christopher, May 30 '20 at 01:10
Does this answer your question? [C# Stringbuilder OutOfMemoryException](https://stackoverflow.com/questions/7537795/c-sharp-stringbuilder-outofmemoryexception) — rph, May 30 '20 at 01:18
Check Task Manager and see how much memory you have. I had similar issue this year and the fix was to convert project from x86 to 64 bit. Task Manager indicated I used all the memory on a 8GB machine. I then borrowed a co-worker machine with 32GB and it got OOM while task manager showed I was only using 14GB. Changing project option to x64 solved issue. — jdweng, May 30 '20 at 01:18
@jdweng sir, I already tried setting target platform X64 bit in VS2010 but it is giving one more exception like "Project assembly was failed to load". project is running fine when set to X86 and ANYCPU — Hussain Md, May 30 '20 at 01:21
@jdweng I checked TaskManager and totally 26% of Momory used — Hussain Md, May 30 '20 at 01:28
Note that when you concatenate all three strings together with "+" before calling sb.Append, that you are briefly consuming the entire space for the three strings together in a single block of memory, then calling sb.Append which will add another equal-sized block of memory (on average) to the StringBuffer while still holding a reference to the first via its parameter, and then only after the Append method returns will the first block be released for GC. It would probably be better to have three Append calls for each of the pieces instead of concatenating them before calling Append. — Some Guy, May 30 '20 at 01:41
Try to make your 32 bit app [LARGEADDRESSAWARE](https://learn.microsoft.com/en-us/cpp/build/reference/largeaddressaware-handle-large-addresses?view=vs-2019). Could solve the problem. See [How to make a .NET application large address aware?](https://stackoverflow.com/q/1346480/10216583) to know how. Also, consider the first comment here. — , May 30 '20 at 01:42
... and don't do that `sb.Append(pdfString + "
" + sbFooter.ToString());` . Read @SomeGuy comment. — , May 30 '20 at 02:01
*My machine is 64 bit* makes absolutely no difference if your code is 32-bit. 32-bit code is limited to 2 GB of total memory unless you've enabled LARGEADDRESSAWARE as @JQSOFT mentioned (which IIRC extends it to 3 GB). Your machine could be 64-bit with 12 TB of RAM and it wouldn't change the limits allowed by 32-bit code. — Ken White, May 30 '20 at 02:18
Additionally, it would be better to preallocate the StringBuffer in the second code block to exactly the size you need at its construction time (calculate combined size of strings you will append to it and pass that to its constructor), rather than letting the StringBuffer try to anticipate how much space you might ever need as you append to it, since otherwise it will probably try to add extra unnecessary capacity. (See first answer [here](https://stackoverflow.com/questions/10196942/how-much-to-grow-buffer-in-a-stringbuilder-like-c-module/10197167#10197167) for details.) — Some Guy, May 30 '20 at 02:24
I also suggest that you take a look at the images you are putting in your PDF and try to determine if you are using a higher resolution than you really need to use, given the size at which you are displaying them in the PDF. If the resolution is working out to more than about 300 pixels per inch, you are probably putting more detail in the document than the human eye can really perceive. Look [here](https://www.adorama.com/alc/0008392/article/100-in-100-Size-Matters) for some discussion of normal print resolutions and effects on file size. Using Base64 will make these worse, of course. — Some Guy, May 30 '20 at 02:32
Need to run in 64 mode. As you can see you are having similar issue that I had. You have plenty of memory but getting OOM. Try a clean build with 64 bit mode. Check bin folder and see if there are any dll files that are reference from other projects. These dll need to be 64. You may want to open the csproj file with Notepad to see why you are having build issues. — jdweng, May 30 '20 at 09:01

score 0 · Answer 1 · answered Jun 01 '20 at 12:31

.NET has memory limits depending on what binarity you run in. And ones that are leagues short of the OS limits, neveremind the hardware limits: https://learn.microsoft.com/en-us/windows/win32/memory/memory-limits-for-windows-releases

Plus the 2 GiB object limit. And strings or arrays (wich I am betting string builder uses) run into the object size or memory limits notoriously quickly.

By default all .NET code is bitness agnostic. The same code can run as x32 or x64 - or even x16 or x128, if we ever get a runtime for that. Fixing the process to only run on x64 binarity can fix the immediate problem. However the bigger problem is the process itself:

Strings are easily capable of breaking the Memory limits. But they got nothing on Images. Images are huge! They are the book example for a BLOB - Binary Large OBject - a scary thing to keep in memory.

Base64 encoding them makes them about 33% bigger. If you then try to keep a whole PDF document of such giant strings in memory, you can only run into issues. It is a wonder it worked thus far. You need to change your process, so you get way smaler strings in memory. Wich may actually mean a whole lot less concatenation, wich may even invalidate having to use StringBuilder in the first place.

Why StringBuilder can't hold huge strings and throwing OOM Exception

1 Answers1