I am working on a report generation windows service which will basically fetch the data from DB, create excels in memory and then will save the excels as byte[] in DB.
To speed up things, I decided to use TPL and below is some code which I have written.
List<ReportData> lstRd = new List<ReportData>();
List<Task> tasks = new List<Task>;
foreach (var item in lstRd)
{
tasks.Add(Task.Factory.StartNew(GenerateReport()));
}
Task.WaitAll(tasks.ToArray());
tasks.ForEach(x=>
if(!x.IsCompleted)
x.Wait());
Task.Factory.ContinueWhenAll(tasks.ToArray(),StopService());
GenerateReport() method will get the data from DB for each report and create the excel and save the byte[] in db. So basically this is time taking and memory intensive operation.
The Problem
My problem is that whenever I see task manager in the server, the process never seems to free up the memory used. Sometimes when one task completes (I assume it completes when the flag is updated as processed in DB) and the other one is waiting to get the data from DB, the memory used in task manager stays the same and as the second task starts processing the memory used keeps on growing.
I was expecting that TPL will automatically Dispose() the task once it completes and GC will free up the memory. But the memory keeps on being used and ultimately I get out of memory exception.
Am I missing something or doing something wrong? It is difficult for me to post the entire code of generate report method but if needed I will try to post some of it.
