C# not releasing memory after task complete

Question

The following code is a simplified example of an issue I am seeing. This application consumes approx 4GB of memory before throwing an exception as the dictionary is too big.

 class Program
 {
    static void Main(string[] args)
    {
        Program program = new Program();

        while(true)
        {
            program.Method();
            Console.ReadLine();
        }
    }

    public void Method()
    {
        WasteOfMemory memory = new WasteOfMemory();
        Task tast = new Task(memory.WasteMemory);
        tast.Start();
    }


}

public class WasteOfMemory
{
     public void WasteMemory()
     {
         Dictionary<string, string> aMassiveList = new Dictionary<string, string>();

         try
         {
             long i = 0;
             while (true)
             {
                 aMassiveList.Add(i.ToString(), "I am a line of text designed to waste space.... I am exceptionally useful........");
                 i++;
             }

         }
         catch(Exception e)
         {
             Console.WriteLine("I have broken myself");
         }
     }
}

This is all as expected, although what we cannot currently work out is when this memory should be released from the CLR.

We have let the task complete and then simulated a memory overload situation, but the memory consumed by the dictionary is not released. As the OS is running out of memory, is it not putting pressure on the CLR to release the memory?

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

So, why is the memory not being released? And how can we get the CLR to release the memory?

Any explanations or solutions would be greatly appreciated.

EDIT: Following replies, particularly Beska's, it is obvious my description of the issue is not the the clearest, so I will try to clarify.

The code may not be the best example, sorry! It was a quick crude piece of code to try to replicate the issue.

The dictionary is used here to replicate the fact we have a large custom data object, which fills a large chunk of our memory and it is not then released after the task has completed.

In the example, the dictionary fills up to the limit of the dictionary and then throws an exception, it does NOT keep filling forever! This is well before our memory is full, and it does not cause an OutOfMemoryException. Hence the result is a large object in memory, and then the task completes.

At this point we would expect the dictionary to be out of scope, as both the task and the method 'Method' have completed. Hence, we would expect the dictionary to be garbage collected and the memory reclaimed. In reality, the memory is not freed until 'Method' is called again, creating a new WasteOfMemory instance and starting a new task.

Hopefully that will clarify the issue a bit

wait.. you just wrote a program meant to occupy the whole memory. What are you complaining about? The methods don't ever return. The scope of the dictionaries never finish so they are never released. Everything is how it's meant to be. — Andre Pena, Jul 03 '12 at 17:12
I am probably missing something but why would the CLR release the memory. The dictonary is a local variable in the method WasteOfMemory, you are still using it, so the GC (Garbage Collector which manages memory in the framework) will not go near it. It seems it is still in scope and says "hey that guy WasteofMemory is still running and using aMassiveList". When you run out of memory it throws an exception. This makes sense. It cannot dispose of it to release the memory as it is still in use. — Michael Hollywood, Jul 03 '12 at 17:13
FYI the `string` being added to the dictionary will probably be interned so `WasteMemory` will leak more slowly than you're expecting. — Lee, Jul 03 '12 at 17:14
@ Daniel A. White - Yes, I have read the description on MSDN and their blogs. I have also tried using dotTrace to profile the memory (without the best luck) but it seemed to show the objects had been collected, but the memory still seemed to be held by the CLR. — DanGordon, Jul 03 '12 at 17:14
@ Andre Pena - the method WasteMemory() fills the dictionary (about 4GB) then throws the exception, hence the task ends. Is this not right? — DanGordon, Jul 03 '12 at 17:21
@ Michael Holywood - is this because the object 'memory' is still in scope within 'Method'? I would have thought that once 'Method' had spun off the task, it would return, hence once the task had ended it would be out of scope, or am I well off the mark? — DanGordon, Jul 03 '12 at 17:27
@DanGordon : I think that you're trying to do something that isn't super-obvious at first...let me know if I've rephrased the crux of your question correctly in my answer. (If I've missed the boat, I'll just delete it, so as not to muddy the waters.) — Beska, Jul 03 '12 at 19:44
Have you tried to ask GC to reclaim memory..http://msdn.microsoft.com/en-us/library/xe0c2357.aspx — Osama Javed, Jul 04 '12 at 09:04
@ Osama Javed - yes we have and it did not free the memory up. We have now implemented a solution similar to [this](http://stackoverflow.com/questions/823661/explicitly-freeing-memory-in-c-sharp). However that feels like we are forcing it rather than getting to the root of why the memory is not being released. — DanGordon, Jul 04 '12 at 10:21

marc wellman · Answer 1 · 2012-07-03T17:21:50.937

6

The garbage collector only frees locations in memory that are no longer in use that are objects which have no pointer pointing to them.

(1) your program runs infinitely without termination and

(2) you never change the pointer to your dictionary, so the GC has certainly no reason to touch the dictionary.

So for me your program is doing exactly what it is supposed to do.

edited Jul 03 '12 at 17:21

answered Jul 03 '12 at 17:15

marc wellman

5,808
5
32
59

score 2 · Answer 2 · answered Jul 03 '12 at 17:15

The memory is not being released because the scope aMassiveList is never finished. When a function returns, it releases all non-referenced resources created inside it.

In your case, aMassiveList never leaves context. If you want your function never to return you have to find a way to 'process' your info and release it instead of storing all of them forever.

If you create a function that increasingly allocates resources and never release it you will end up consuming all the memory.

score 2 · Answer 3 · answered Jul 03 '12 at 17:18

2

GC will only release unreferenced objects, so as the dictionary is being referenced by your program it can't be released by the GC

answered Jul 03 '12 at 17:18

R Quijano

1,301
9
10

score 2 · Accepted Answer · answered Jul 03 '12 at 18:38

Okay, I've been following this...I think there are a couple issues, some of which people have touched on, but I think not answering the real question (which, admittedly, took me a while to recognize, and I'm not sure I'm answering what you want even now.)

This is all as expected, although what we cannot currently work out is when this memory should be released from the CLR.

As others have said, while the task is running, the dictionary will not be released. It's being used. It gets bigger until you run out of memory. I'm pretty sure you understand this.

We have let the task complete and then simulated a memory overload situation, but the memory consumed by the dictionary is not released. As the OS is running out of memory, is it not putting pressure on the CLR to release the memory?

Here, I think, is the real question.

If I understand you correctly, you're saying you set this up to fill up memory. And then, after it crashes (but before you hit return to start a new task) you're trying other things outside of this program, such as running other programs in Windows to try to get the GC to collect the memory, right? Hoping that the OS would talk to the GC, and start pressuring it to do it's thing.

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

I think you answered your own question...it has not been necessarily been released until you hit return to start a new task. The new task needs memory, so it goes to the GC, and the GC happily collects the memory from the previous task, which has now ended (after throwing from full memory).

So, why is the memory not being released? And how can we get the CLR to release the memory?

I don't know that you can force the GC to release memory. Generally speaking, it does it when it wants (though some hacker types might know some slick way to force its hand.) Of course, .NET decides when to run the GC, and since nothing is happening while the program is just sitting there, it may well be deciding that it doesn't need to. As to whether the OS can pressure the GC to run, it seems from your tests the answer is "no". A bit counter-intuitive perhaps.

Is that what you were trying to get at?

I think you are on the right track, this is more what I was trying to get to the bottom of and it seems a bit like there may be nothing we can do. I will edit the question above to make it clearer and reassess based on the replies so far. — DanGordon, Jul 03 '12 at 21:16
Chosen as answer as whilst it did not get to the root of the issue elegantly explained what is happening and confirmed my suspicion that the GC seems to just collect when it wants to. It was possible to "force" the GC into a clean up by clearing the dictionary, setting it to null and calling GC collect as seen in this question: http://stackoverflow.com/questions/823661/explicitly-freeing-memory-in-c-sharp — DanGordon, Jul 06 '12 at 11:03

score 0 · Answer 5 · answered Jul 03 '12 at 17:49

The way you've written the WasteMemory method, it will never exit (unless the variable "i" overflows, which won't happen this year) and BECAUSE IT WILL NEVER EXIT it will keep IN USE the reference to the internal Dictionary.

Daniel White is right, you should read about how GC works.

If the references are in use, GC will not collect the referenced memory. Otherwise, how would any program work?

I don't see what you expect the CLR/GC to do here. There's nothing to garbage-collect inside one run of your WasteMemory method.

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

When you press Enter, a new task is created and started. It's not the same task, it's a new task - a new object holding a reference to a new WasteOfMemory instance.

The old task will keep running and the memory it uses will NOT be collected because the old task keeps running in background and it keeps USING that memory.

I'm not sure why - and most importantly HOW - you observe the memory of the old task being released.

score -8 · Answer 6 · edited May 19 '14 at 22:04

Change your method to be a using statement

Example:

Using (WateOfMemory memory = new WateOfMemory())
{
    Task tast = new Task(memory.WasteMemory); 
    tast.Start();
 }

And add disposible WateOfMemoryClass (by the way your constructor is WasteOfMemory)

#region Dispose
    private IntPtr handle;
    private Component component = new Component();
    private bool disposed = false;

    public WateOfMemory()
    {

    }

    public WateOfMemory(IntPtr handle)
    {
        this.handle = handle;
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    private void Dispose(bool disposing)
    {
        if(!this.disposed)
        {
            if(disposing)
            {
            component.Dispose();
            }

            CloseHandle(handle);
            handle = IntPtr.Zero;            
        }
        disposed = true;         
    }

    [System.Runtime.InteropServices.DllImport("Kernel32")]
    private extern static Boolean CloseHandle(IntPtr handle);

    ~WateOfMemory()      
    {
        Dispose(false);
    }
    #endregion

You're also disposing WasteOfMemory before the task has completed. The task is running asynchronously, so the correct way to dispose (given this design) would be with a continuation. — Kent Boogaart, Jul 03 '12 at 17:19

C# not releasing memory after task complete

6 Answers6

Linked