OutOfMemoryError calling XmlSerializer.Deserialize() - not related to XML size!

Question

This is a really crazy bug. The following is throwing an OutOfMemoryException, for XML snippits that are very short and simple (e.g., <ABC def='123'/>):

public static T DeserializeXmlNode<T>(XmlNode node)
{
    try
    {
        return (T)new XmlSerializer(typeof(T))
            .Deserialize(new XmlNodeReader(node));
    }
    catch (Exception ex)
    {
        throw; // just for catching a breakpoint.
    }
}

I read in this MSDN article that if I were using XmlSerializer with additional parameters in the constructor, I'd end up generating un-cached serializer assemblies every time it got called, causing an Assembly Leak. But I'm not using additional parameters in the constructor. It also happens on the first time it is called in a freshly started AppDomain, so that doesn't make sense either.

What gives?

Answer 1

Well, the final answer to my question isn't going to help everyone that encounters this, but some of my coworkers also encountered this months later on a different system with a different product. The laughed when they found my post here on SO months later and wondered if I actually had solved it or not, since no solution was accepted here.

The final solution has nothing to do with problems Deserializing. Instead, it involved completely uninstalling and installing a brand new copy of Oracle ODP.NET database client, the provider many if not all of our applications use.

Based on anecdotal evidence, it seems this problem arises on improperly patched versions of ODP.NET assemblies, of which subsequently got propagated to other systems via virtual machine clones.

When ODP.NET was completely removed, and a new compatible version was retrieved from the Oracle website and installed, the problem disappeared completely.

The hypothesis is that a usable (but corrupt) ODP.NET driver has unsafe code and was repeatably overwriting the .NET protected memory area near the Deserialize method after first use. If Deserialize was called before any ODP.NET invocations, it would work just fine. However, all subsequent calls to Deserialize after using any ODP.NET calls would fail miserably.

The final solution to this that has now been resolved twice in two distinct products is to get a good/fresh/clean/new copy of ODP.NET installed.

Not pretty... but that's what solved it.

Answer 2

EDIT: This was not the solution, unfortunately, but it may help others track down a very similar problem. This answer here is the actual solution.

I've believe I found the solution to this problem. It is a bug in .NET 3.5 SP1.

Serialization hangs or throws an OutOfMemoryException with static delegate and ISerializable on 3.5 SP1 (ID: 361615):

When a generic class implements ISerializable and has a static delegate member that makes use of the generic type arguments, binary deserialization hangs (on a 32-bit system with Windows Server 2003) or throws an OutOfMemoryException (on a 64-bit system with Windows Server 2008).

This error occurs with .NET 3.5 SP1 and did not occur with .NET 3.5 without SP1.

The solution is to install KB957543 hot fix.

Answer 3

You haven't provided enough details to recreate your problem. But, the reader implements IDisposable and should be disposed of properly. Preferably by wrapping it in a using block. Most developers never run into a problem when they forget to dispose of something because the garbage collector will eventually clean up the mess. However, it isn't hard to code something that causes problems before the GC gets around to cleanup, or even prevents cleanup entirely.

public static T DeserializeXmlNode<T>(XmlNode node)
{
    XmlSerializer xs = new XmlSerializer(typeof(T));
    using(XmlNodeReader xr = new XmlNodeReader(node))
        return (T)xs.Deserialize(xr);
}

Answer 4

Based on the documentation of the XmlSerializer class you should cache the XmlSerializers otherwise you can cause poor performance or a memory leak.

Hashtable serializers = new Hashtable();

// Use the constructor that takes a type and XmlRootAttribute.
XmlSerializer s = new XmlSerializer(typeof(MyClass), myRoot);

// Implement a method named GenerateKey that creates unique keys 
// for each instance of the XmlSerializer. The code should take 
// into account all parameters passed to the XmlSerializer 
// constructor.
object key = GenerateKey(typeof(MyClass), myRoot);

// Check the local cache for a matching serializer.
XmlSerializer ser = (XmlSerializer)serializers[key];
if (ser == null) 
{
    ser = new XmlSerializer(typeof(MyClass), myRoot);
    // Cache the serializer.
    serializers[key] = ser;
}
else
{
    // Use the serializer to serialize, or deserialize.
}