37

I've create a c# application which uses up 150mb of memory (private bytes), mainly due to a big dictionary:

Dictionary<string, int> Txns = new Dictionary<string, int>();

I was wondering how to free this memory up. I've tried this:

Txns = null;
GC.Collect();

But it doesn't seem to make much of a dent in my private bytes - they drop from say 155mb to 145mb. Any clues?

Thanks

-edit-

Okay I'm having more luck with this code (it gets the private bytes down to 50mb), but why?

Txns.Clear(); // <- makes all the difference
Txns = null;
GC.Collect();

-edit-

Okay for everyone who says 'don't use GC.collect', fair enough (I'm not going to debate that, other than saying you can see my C background coming through), but it doesn't really answer my question: Why does the garbage collector only free the memory if i clear the transaction list first? Shouldn't it free the memory anyway, since the dictionary has been dereferenced?

akjoshi
  • 15,374
  • 13
  • 103
  • 121
Chris
  • 39,719
  • 45
  • 189
  • 235
  • GC.Collect() doesn't enforce that all memory that could be collected is collected at that time, as the garbage collection is a multi-step process. A process I might add that you usually do not need to even know the details about... – peSHIr May 05 '09 at 07:05
  • Is it necessary to set Txns to null, will it make any difference? or just Clearing the content is enough(to free memory)? – akjoshi Dec 06 '10 at 12:00
  • There are multiple levels of clearing memory. GC is complicated piece of... look at source code file if you want. Anyway it can run multiple levels of searching for cleanable blocks of memory. Entries in dictionary are still referenced by dictionary, so they are referenced. GC would need to check if all references are referenced, what would take much longer... often shallow search is enough. And if it's enough don't eat more user time assumption comes. – Adas Lesniak Feb 24 '22 at 13:02

8 Answers8

19

Private bytes reflect the process' memory usage. When objects are collected the associated memory segment may or may not be freed to the OS. The CLR manages memory at the OS level and since allocating and freeing memory isn't free there's no reason to free each piece of memory immediately as chances are that the application will probably request more memory later.

Brian Rasmussen
  • 114,645
  • 34
  • 221
  • 317
  • 3
    That all sounds reasonable, but it really _does_ free the memory to the OS if i call Txns.Clear(). Plus, i really want to be nice to the OS memory, since i'm not the only one using this server. – Chris May 05 '09 at 23:07
  • 2
    I guess what i'm asking is, why does txns.Clear() give memory back to the OS, when deferencing the entire dictionary doesn't give it back? – Chris May 05 '09 at 23:08
  • 4
    The point is that in a managed application you're one step removed from dealing directly with OS memory as the CLR does that for you. You may be able to construct scenarios where you can observe a direct effect as you describe, but that is not guaranteed to work as you depending on an implementation details of the CLR. – Brian Rasmussen May 06 '09 at 03:33
12

If you call GC.Collect() it starts to do its job, but returns immediately instead of blocking so you don't see any effect. If you just call GC.WaitForPendingFinalizers() it will block your app until GC.Collect() has finished its job.

Clonkex
  • 3,373
  • 7
  • 38
  • 55
Sadegh
  • 6,654
  • 4
  • 34
  • 44
  • 1
    Yikes! Can't even begin to think about the levels of badness of starting a thread that periodially (and rather often: it tries twice a second!) calls GC.Collect and then waits for all pending finalizers. Are you just totally not interested in scaling applications? That's just trying to hide a bad symptom, badly! Guess I'll just have to -1 here... – peSHIr May 05 '09 at 07:09
  • 11
    @peSHIr: agreed, though to be fair, it does explicitly answer the question. Whether is a good idea or not is a different question. – andy May 05 '09 at 07:15
  • @andy: begrudgingly agreed. Should I -1 the question instead then...? Also seems unnecessarily harsh and useless. Oh well. – peSHIr May 05 '09 at 07:17
  • I'm not sure that is entirely correct. A garbage collection will (may?) wake up the finalizer thread, which will then run finalizers as needed. Thus waiting for the finalizer is not really waiting for the garbage collection to finish (as GC happens on a different thread) – Brian Rasmussen May 05 '09 at 08:27
  • Hi, Thanks for this answer but it doesn't seem to make a difference, as i tried this and the private bytes didn't go down: //Txns.Clear(); Txns = null; for (int x = 0; x < 20; x++) { GC.Collect(); GC.WaitForPendingFinalizers(); } Now keep in mind everyone this is proof of concept code! Its not going into production. But the only way i can get it to free the memory is calling the dictionary's Clear method, which i _really shouldn't_ have to call, since i'm dereferencing the whole dictionary. – Chris May 05 '09 at 23:05
  • Peshlr - who mentioned starting threads and running the GC twice a second? Is that an earlier revision of the above answer? – Chris May 05 '09 at 23:21
  • I'm no expert on the GC, but I actually ran into a scenario where I needed to do just this (at least it fixed the problem I was having). I used it to free up Com objects. Calling both Collect() and WaitForPendingFinalizers() on the GC fixed the problem. In all my years of programming, I've never had to explicitly gc.Collect() until now. – goku_da_master Oct 30 '11 at 05:45
5

Most probably you have a hidden reference to the dictionary somewhere else. Thus the dictionary is not collected, but if you Clear() it, the contents get collected.

As others have already noted, forcing the GC is not recommended. This might lead to memory being pushed into higher "generations" which are not often collected, thereby wasting more memory than gained in the long run.

David Schmitt
  • 58,259
  • 26
  • 121
  • 165
  • Nope, no hidden references to the dictionary. – Chris May 05 '09 at 22:44
  • 1
    Your comment about pushing objects into higher generations, care to elaborate? That sounds like an issue to be aware of. – Chris May 05 '09 at 22:57
  • Hang on, i thought that if you call gc.collect with no args, it simply collects _all_ generations? Wouldn't that rule out this issue with pushing stuff into higher generations? Thanks. – Chris May 05 '09 at 23:01
  • GC.Collect() does collect all gens. There are two issues here. 1) call it once: still referenced memory is pushed up, thus delaying normal collection, and 2) calling it regularly wastes more cpu because the gc has to process all gens everytime. So whatever you do with GC.Collect(), you loose. Except, of course, in those special cases where those effects don't matter to you. – David Schmitt May 06 '09 at 07:14
  • Running a memory profiler at this stage would have proven no hidden reference exist. – crokusek Mar 02 '15 at 19:50
4

Not sure from memory if Dictionary has a Dispose() or not, but it's bound to have a Clear(). Call either of these before setting any references to null.

Then, simply let the garbage collector do its work. It is almost never a good idea to call GC.Collect() explicitly yourself and it might not even do what you want/need/expect and end up costing you performance. Static Code Analysis (=FxCop) doesn't warn you with Reliability rule CA2001 about this for nothing, you know? Simply do not do this unless you really know what you're doing. And even then don't do it. ;-)

Are you sure the dictionary is that huge? Isn't it just 10 Mb of memory and the rest is otherwise taken by your application? Question that might help you: Have you used a profiler yet to see where memory is actually consumed...?

peSHIr
  • 6,279
  • 1
  • 34
  • 46
  • The dictionary *is* that big. When i ran the code with the .Clear change, memory usage goes from 160mb to 50mb. So i assume the dictionary is 110mb. – Chris May 05 '09 at 22:58
1

Ok, I have a theory here... Dictionary is a collection of KeyValuePair which is again a reference type.

Your dictionary contains these keyValuePairs. When you say:

Txns = null

It frees the reference 'Txns' from those KeyValuePair collection. But still the actual memory of 150 Mb is being referenced by those KeyValuePair and they are in scope, thus not ready for garbage collection.

But when you use the following:

Txns.Clear();
Txns = null;
GC.Collect();

Here, the clear method also frees the 150Mb of data from their respective KeyValuePair object references. Thus those objects were ready for garbage collection.

Its just a wild guess I am making here. Comments are welcome :)

Savaratkar
  • 1,974
  • 1
  • 24
  • 44
  • The KeyValuePairs aren't in scope. To have them in scope, you would have to setup up additional variables or an array to hold copies of the KeyValuePairs, similar to this: `var arrayOfReferencesToKeyValuePairs = Txns.ToArray();` In his scenario, once `Txns` is null, there is no way to access any of the KeyValuePairs and the garbage collector sees that. – jnm2 Apr 08 '15 at 19:16
1

Edit:

To be fair, setting the reference to null does not free the memory, it assigns it's container to a different address, in this case null. According to MSDN, calling Clear(), does this: "The Count property is set to 0, and references to other objects from elements of the collection are also released. The capacity remains unchanged."

...

You should never call the garbage collector. You are using managed objects with no native resources, trust the garbage collector to clean up after you.

Besides the size of your dictionary, you don't need to worry about memory, memory isn't your problem, it's garbage collectors problem.

Calling Clear() will remove the references to any contained object inside, but the capacity remains unchanged.

On a technical note, collecting memory is expensive and a considerable time consuming operating. Reason being, not only does the GC handle heap memory and clean up your heap, it also sort of defrags the heap. It tries to move memory into contiguous blocks to speed up allocation for when some piece of code makes a large request.

p.s. How big is your dictionary that you use 155MB of memory?

Chris
  • 6,702
  • 8
  • 44
  • 60
  • Its big, i'm reconciling a million transactions. – Chris May 05 '09 at 22:45
  • holy crap that is a lot of transactions. I'm assuming this is a one time thing, or do you need to do this regularly? – Chris May 06 '09 at 03:19
  • Actually i got that wrong - about half a dozen times each day. – Chris May 07 '09 at 00:20
  • OK, if that's the case, is there a reason it must be done in code? Why not just use the database to process these transactions, is there a reason not to? – Chris May 07 '09 at 20:54
1

Do you need the memory back? The memory is available, it's just not being reclaimed. You should not clear the dictionary, hold a weak reference to it and let the runtime do its job.

If you want to see really deep into what's going on, check out the .NET Memory Profiler. That will give you visibility into exactly what's happening with your object, what generation it is, what memory is being used by what and on and on. :)

JP Alioto
  • 44,864
  • 6
  • 88
  • 112
  • I don't *need* the memory back, however in our situation i'd prefer to give it back to the OS quickly, if possible. I guess this question is becoming more of a theoretical 'how to do this?' rather than a 'i really need to do this' thing... – Chris May 07 '09 at 00:19
0

Windows has two memory availability events. I'd expect the CLR to respond to this. If there's plenty of memory available, the smart thing is to NOT run the garbage collector. So, to make sure that you are indeed observing bad CLR behavior, please repeat this test with another dummy application using a big heap of memory.

MSalters
  • 173,980
  • 10
  • 155
  • 350