EDIT
The good news is that the weird behavior explained below is not related to the ConcurrentBag, that is: Threads related to the concurrent bag are eventually freed up. Yet Threads themselfs are kept alive for some or other reason. In the example code given i clearly create a thread and destroy all references. Yet a Garbage collection does not pick it up. In fact, the only moments i have discovered so far for when threads gets totally destroyed are when the concurrent bag itself gets collected (if i dont collect the concurrent bag, the Threads will stay alive), or when we create a number of other threads.
ORIGINAL (The original problem and some motivation. The sourcecode below this part explains the important aspects)
This is a replicate of a question i asked months before with regards to the ConcurrentBag ( Possible memoryleak in ConcurrentBag? ). Appareantly, the ConcurrentBag doesn't behave as it should be and now i'm worried about the stability of some running legacy code. This question comes in a response to my findings when answering the following question: How can I free-up memory used by a Parallel.Task?
The scenario hasnt changed a bit. I have a webservice which handles messages. Clients can shoot in messages using some public API. These requests will be handeled by a Thread from the .NET ThreadPool which in turn adds the message to a ConcurrentBag. Next, there are concurrent tasks consuming messages from the ConcurrentBag and handling them. there are peak hours with many messages being added and many messages being consumed and there moments when nobody is doing anything. Soo, the threadpool itself is subjected to change its amount of active threads quite extensively during the running time (desired running time is 'forever').
Now it turns out that as soon as a thread calls ConcurrentBag.Add (or any method on the ConcurrentBag). the thread is kept alive by a reference internally held in the ConcurrentBag and is only released when the concurrentBag itself is actually cleaned up by the GC. In my scenario, this will lead the a infinite amount of 'waste' threads who are not cleaned up over time since the ConcurrentBag is alive throughout the application's lifetime.
The previously given solution of simply emptying the bag doesn't help either since there is not the problem. The problem (Assumably is that ConcurrentBag doesn't call Dispose on the ThreadLocal it holds for the current Thread since there is no way for the ConcurrentBag to know when a thread is ends). (Yet it should do cleanup whenever you access the bag again).
That said, should we stop using ConcurrentBag in most cases or are there solutions i can add to work around this problem?
My test code:
Action collectAll = () =>
{
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
};
ConcurrentBag<object> bag = new ConcurrentBag<object>();
Thread producerThread = new Thread(new ThreadStart(delegate
{
// produce 1 item
bag.Add(new object());
}));
// create a weakreference to the newly created thread so we can track its status
WeakReference trackingReference = new WeakReference(producerThread);
// start the thread and let it complete
producerThread.Start();
producerThread.Join();
// thread can now be set to null, after a full GC collect, we assume that the thread is gone
producerThread = null;
collectAll();
Console.WriteLine("Thread is still alive: " + trackingReference.IsAlive); // returns true
// consume all items from the bag and collect again, the thread should surely be disposed by now
object output;
bag.TryTake(out output);
collectAll();
Console.WriteLine("Thread is still alive: " + trackingReference.IsAlive); // returns true
// ok, finally remove all references to the Bag
bag = null;
collectAll();
Console.WriteLine("Thread is still alive: " + trackingReference.IsAlive); // returns false
