4

I am getting a nice SystemOutOfMemory exception in my StringBuilders. It is not due to lack of ram, thus I believe it is memory fragmentation.

I have ~200 StringBuiler objects. All of these are reused regularly (with strBldr.clear()). This seems to cause my system to fragment the memory pretty bad. How can I solve this?

Thanks :)

EDIT:

Here are some data:

Max recorded size of input& stringBuilder: 4 146 698.

Avarage re-initiated stringBuilders/second: >120 (possibly >>120)

Length of input @ first error: 16 972 (string)

StringBuilder length @ first error: 16

Count of times a new stringBuilder had been made @ first error: ~32500

Total ram usage @ first error 637 448K

Automatico
  • 12,420
  • 9
  • 82
  • 110
  • How exactly do you know it's not due to a lack of RAM? What type of testing/profiling/debugging have you done already? – Cody Gray - on strike May 14 '11 at 10:47
  • I am running on a machine with 16 GB ram, and the program has clocked in at 700 Mb max (Task manager). Thus I conclude it is not lack of ram. – Automatico May 14 '11 at 10:49
  • 2
    Yeah, that's *total* RAM. What about exceeding the amount of memory that is available to allocate to an individual `StringBuilder` instance? Check [Remarks](http://msdn.microsoft.com/en-us/library/system.text.stringbuilder.aspx) on MSDN: "A StringBuilder can allocate more memory as needed to store characters when the value of an instance is enlarged, and the capacity is adjusted accordingly. The amount of memory allocated is implementation-specific, and either ArgumentOutOfRangeException or OutOfMemoryException is thrown if the amount of memory required is greater than the maximum capacity. – Cody Gray - on strike May 14 '11 at 10:51
  • Also see: [StringBuilder for string concatenation throws OutOfMemoryException.](http://stackoverflow.com/questions/363680/stringbuilder-for-string-concatenation-throws-outofmemoryexception), [interesting OutOfMemoryException with StringBuilder](http://stackoverflow.com/questions/1769447/interesting-outofmemoryexception-with-stringbuilder), [Memory free up string builder an d byte in c#: Out of memory exception](http://stackoverflow.com/questions/1733667/memory-free-up-string-builder-an-d-byte-in-c-out-of-memory-exception) – Cody Gray - on strike May 14 '11 at 10:54
  • @Cody, your two comments above would make a good answer, don't you think? :) – Frédéric Hamidi May 14 '11 at 10:55
  • @Frederic: I didn't post them as an answer because they don't really explain how to *solve* the problem. – Cody Gray - on strike May 14 '11 at 10:56
  • I assume that the "max capacity" is the max length it can allocate to the instance. I the current length of the stringBuilder that caused the problem was 12M, a long way away from the max capacity listed. The outOfMemory exception is not a one time event, but rather recurring continuously ones it first starts. - I checked this very moment. The same thing happened with a stringBuilder of 8M length. Max cap is 2 147 M. – Automatico May 14 '11 at 11:05
  • Cort3z, you should post some more information (sizes, usage patterns). – H H May 14 '11 at 11:23
  • Here are some data: Max recorded size of input& stringBuilder: 4 146 698; Avarage new bufferBuilders/second: >120 (possibly >>120); Length of input @ first error: 16 972 (string); StringBuilder length @ first error: 16; Count of times a new stringBuilder had been made @ first error: ~32500; Total ram usage @ first error 637 448K – Automatico May 14 '11 at 12:11

3 Answers3

4

You should not reuse a StringBuilder like that, just create a new one as needed.

When you call Clear on a StringBuilder it will not release all the memory that it used, it will only reset the used size to zero. It will still have the same large buffer, and repeated use of the StringBuilder only means that the buffer will be as large as it ever needed to be, and never shrink.

Also, keeping the StringBuilder objects for reuse means that they survice garbage collections and move on to the next generation heap. Those are collected less frequently, so they are more likely to be sensetive to memory fragmentation.

Guffa
  • 687,336
  • 108
  • 737
  • 1,005
4

I agree, most likely you are not running out of memory but into fragmentation.

You have to get acquainted with fragmentation and the Large Object heap (LOH).

You don't provide any details so I can only give some very broad advice:

  • try to estimate how large your strings will be and use the Capacity parameter for a new SB
  • round up (really) those sizes to multiples of some number. This promotes re-use.
  • only use Clear() when you expect the new contents to be almost the same size as the old, growing it is what kills you.

Edit

Max recorded size of input& stringBuilder: 4 146 698.

  • Make sure there are no intermediates needed with a greater size, then
  • Create all StringBuilders like sb1 = new StringBuilder(4200000);
  • Don't try to re-use them (too much / at all)
  • Don't keep them around too long
H H
  • 263,252
  • 30
  • 330
  • 514
  • When you say: "Don't keep them around too long", do you mean that I should set the stringbuilder to null? – Automatico May 15 '11 at 15:31
  • Again, hard to give such broad advice. Nulling local vars is pointless, nulling a field might be useful. Just keep their scopes as short as possible. – H H May 15 '11 at 16:26
0

What I ended up doing is migrating to x64. This solved my problems.

It is possible that I accually assigned the entire memory space of x86, even though I did not use it all. Migrating to x64 certainly would solve that issue.

Automatico
  • 12,420
  • 9
  • 82
  • 110