c# memory leak on Deserialize dictionary

Question

When I deserialize a dictionary with the method, I have I memory leak. Here the following test to reproduce the problem.

public static Dictionary<TKey, TValue> DeserializeDictionary<TKey, TValue>(this string iSerialization)
    {
        Dictionary<TKey, TValue> dic;

        using (var textWriter = new StringReader(iSerialization))
        {
            XmlSerializer serializer = new XmlSerializer(typeof(Item<TKey, TValue>[]), new XmlRootAttribute() { ElementName = "items" });
            dic = ((Item<TKey, TValue>[])serializer.Deserialize(textWriter)).ToDictionary(i => i.Key, i => i.Value);
            textWriter.Close();
        }
        return dic;
    }

public class Item<TKey, TValue>
    {
        [XmlAttribute]
        public TKey Key;

        [XmlAttribute]
        public TValue Value;
    }

the test :

[TestMethod]
    public void test()
    {
        string test = "<?xml version=\"1.0\" encoding=\"utf-16\"?><items xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"><ItemOfStringString Key=\"mykey\" Value=\"myvalue\" /></items>";

        while(true)
        {
            Dictionary<string, string> tesfezf = test.DeserializeDictionary<string, string>();
        }

    }

Do you know where is the problem ?

EDIT: I use this method in a workerazure role, in a loop (arround 20000), and this fill the memory and throw outofmemory exception.

What evidence do you have that tells you there is a memory leak? — phoog, Mar 04 '15 at 19:03
You have no assertion in your test, could you please detail what you assert is the problem? — Peter Ritchie, Mar 04 '15 at 19:04
@phoog In the task manager I can see the memory usage increased — Julian50, Mar 04 '15 at 19:07
Increase in memory usage is expected in this case, you're deserializing data. CLR will perform the work required, and allocate memory needed to make it happen. — Samir Banjanovic, Mar 04 '15 at 19:08
@PeterRitchie You are right, It's test just to run easyly the loop in see the problem in the task manager. After 2000 loop in the while loop +30mo in the memory — Julian50, Mar 04 '15 at 19:08
@Julian50 The fact that the code uses memory doesn't mean it's leaking memory. It just means it's using a lot of memory. — Servy, Mar 04 '15 at 19:14
@Julian50 the memory may be in use, but it is almost entirely eligible for garbage collection, so it is not a memory leak. — phoog, Mar 04 '15 at 19:16
@Servy but I don't understand, because I don't keep the dic named 'tesfezf', I overide it in each loop, right ? So the GC should't free the memory ?? Just overide and overide always the same thing ?? — Julian50, Mar 04 '15 at 19:19
Precompiled or at least cached serializer should fix the issue with multiple dynamically compiled assemblies consuming all memory... http://stackoverflow.com/questions/678676/precompile-xmlserializers-with-xmlattributeoverrides — Alexei Levenkov, Mar 04 '15 at 19:23
@Julian50 The GC doesn't free the memory except under certain conditions. — phoog, Mar 04 '15 at 19:59
@Julian50 - "Just override and overide always the same thing?" - That is not how it works. In your loop variable tesfezf is a reference pointer. Each time you do a deserialize in your loop it points to a new block of memory that was allocated on the heap (ref type). In your case the pointer/var is re-declared but it would not matter even if you had it outside the loop, the pointer only takes up a limited of space (4 or 8 bytes) on the stack. So each new deserialized is put on the heap and GC will only free it when GC runs. Test it add GC.Collect(); inside your loop and check your memory now. — Igor, Mar 04 '15 at 20:53
@igor thanks for your details answer. I tried with GC.Collect() inside the loop but still the same.... So I did a new test. Instead of serialize a create a new dictionary() and added one item. This new dictionary is ref type as well right ? And in this case I have no problem. So I think there is something wrong with my function deserialize, The GC doesn't want to collect. .. ? — Julian50, Mar 04 '15 at 21:53
@Julian50 - I added an answer after testing your scenario. You are right, there is a leak but not where you thought it was. It is more of a cache that is built on constantly though, not really a memory leak but something you should be aware of when designing your application. Check out the links I posted, they also contain plenty of suggestions on how to work around the problem. — Igor, Mar 05 '15 at 01:10

score 2 · Accepted Answer · edited May 23 '17 at 12:14

It has nothing to do with the deserialization but with the instatiation of the XmlSerializer. This particular constructor overload generates a new temporary assembly which is loaded in the application domain with every call and will never be unloaded. End result is you should cache your XmlSerializer or use a different constructor if you plan on using it many times in the life cycle of the application domain.

You can test it with this bit of code, and you will see your memory use increase.

var root = new XmlRootAttribute() {ElementName = "items"};
var type = typeof (Item<string, string>[]);
while (true)
{
    XmlSerializer serializer = new XmlSerializer(type, root);
    GC.Collect();
}

If you move the block XmlSerializer serializer = new XmlSerializer(type, root); out of the loop in your code and just deserialize in the loop the memory stays about constant.

Here are plenty articles all describing the same problem including a support article on Microsoft's web site.

Microsoft Support KB 886385

Blog on Msdn - .NET Memory Leak: XmlSerializing your way to a Memory Leak

A Memory Leak brought to you by XmlSerializer

Question and Answer on StackOverflow

thanks a lot. Only these contructor works in my case 'XmlSerializer(type) and XmlSerializer(type, defaultNameSpace)' and fix my problem. I mean don't charge the memory. I don't know why my question was downvote so many times... anyway. Thanks you are my superman ;) — Julian50, Mar 05 '15 at 06:41
Glad I could help. I am not sure why either but I will +1 it to help you out. — Igor, Mar 05 '15 at 11:52

CharithJ · Answer 2 · 2015-03-05T00:35:09.337

There is no memory leak as far as I see. But there is an obvious bad memory usage in your test case.

CG.Collect() is not imminent most of the time. Specially when there are non-managed resources, it has to clear finalization queue before releasing all the unwanted memory.

In this case it consumes a lot of memory and it doesn't allow Garbage collector to finish its finalization process.

So, after Deserialize, you can call GC.Collect() and wait until GC finish its finalization.

   //Force garbage collection.
   GC.Collect();

   // Wait for all finalizers to complete before continuing. 
   // Without this call to GC.WaitForPendingFinalizers,  
   // the worker loop below might execute at the same time  
   // as the finalizers. 
   // With this call, the worker loop executes only after 
   // all finalizers have been called.
   GC.WaitForPendingFinalizers();

Refer :

GC.WaitForPendingFinalizers

GC.WaitForFullGCComplete Method

c# memory leak on Deserialize dictionary

2 Answers2