XmlSerializer : Generated File too big

Question

I have Problems with the XML Serializer in C#. My generated Files (out of a linked List Point Cloud) is getting Sizes of 30-40 MB. (60.000 Vertices).

Most of the time my Visual Studio crashes (vshost32.exe didnt work anymore) Sometimes im lucky and getting this graphic issue instead:

enter image description here

If i redraw my Model (im using an opengl control) its suddenly displayed how it should look. enter image description here

Im guessing that my memory causes this. Is there anyway a possibility to decrease the size of the xml files? Hes using a whole line for every Tag. (460.000 Lines! )

  <DistanceNeighbours />
  <Vector>
    <X>-8.52</X>
    <Y>51.05</Y>
    <Z>62.56</Z>
  </Vector>

Here are my Serializable + Deserializable Functions, hope you can help me.

     /// <summary>
    /// Serializes an object.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="serializableObject"></param>
    /// <param name="fileName"></param>
    public void SerializeObject<T>(T serializableObject, string fileName)
    {
        if (serializableObject == null) { return; }

        try
        {
            XmlDocument xmlDocument = new XmlDocument();
            XmlSerializer serializer = new XmlSerializer(serializableObject.GetType());
            using (MemoryStream stream = new MemoryStream())
            {
                serializer.Serialize(stream, serializableObject);
                stream.Position = 0;
                xmlDocument.Load(stream);
                xmlDocument.Save(fileName);
                stream.Close();
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("Serialize Object Error");
        }
    }

    /// <summary>
    /// Deserializes an xml file into an object list
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="fileName"></param>
    /// <returns></returns>
    public T DeSerializeObject<T>(string fileName)
    {
        if (string.IsNullOrEmpty(fileName)) { return default(T); }

        T objectOut = default(T);

        try
        {
            string attributeXml = string.Empty;

            XmlDocument xmlDocument = new XmlDocument();
            xmlDocument.Load(fileName);
            string xmlString = xmlDocument.OuterXml;

            using (StringReader read = new StringReader(xmlString))
            {
                Type outType = typeof(T);

                XmlSerializer serializer = new XmlSerializer(outType);
                using (XmlReader reader = new XmlTextReader(read))
                {
                    objectOut = (T)serializer.Deserialize(reader);
                    reader.Close();
                }

                read.Close();
            }
        }
        catch (Exception ex)
        {
            //Log exception here
        }

        return objectOut;
    }

Thank you,

if you want to reduce the size of xml, you have very few options: shorten the tag names, or restructure the xml to be more compact, e.g. `` would require less string space. — Marc B, Jul 07 '15 at 14:57
Why are you using a `MemoryStream` for your serializer, but a `TextReader` for your deserializer? — Robert Harvey, Jul 07 '15 at 14:59
http://stackoverflow.com/questions/7671958/reading-large-xml-documents-in-net — ken lacoste, Jul 07 '15 at 15:47
according to even Microsoft, `XmlTextReader` is obsolete. Provided you're using at least .NET 2.0, you should be looking at the XmlReader class instead. Source: https://msdn.microsoft.com/en-us/library/system.xml.xmltextreader(v=vs.110).aspx EDIT: didn't notice someone already pointed this out, but source is still good. — user2366842, Jul 07 '15 at 15:48
You can write directly to a file using `XmlSerializer`, you don't need the intermediate `XmlDocument`. See [How to: Write Object Data to an XML File](https://msdn.microsoft.com/en-us/library/ms172873.aspx). And similarly: [How to: Read Object Data from an XML File](https://msdn.microsoft.com/en-us/library/ms172872.aspx). — dbc, Jul 07 '15 at 18:32

score 1 · Answer 1 · answered Jul 07 '15 at 15:48

What about binary serialization?

 BinarySerializer serializer = new BinarySerializer (outType);

BinarySerializer is intended to reduce problems of file size and computation needs.

You can store result in a file, in a DB, send trhough the net.

You will need a Binary Deserializer to have back your objects.

score 0 · Answer 2 · answered Jul 07 '15 at 15:23

When working with such large files (and even smaller) and if you do not need to deal with XmlDocument object by adding nodes, validating schema, applying XPath, consider please using simple XmlReader instead.

Just instantiate XmlReader class and call XmlReader.Read() method. You will have to define token type after every read by checking XmlReader.NodeType property. Once you find a specific element with attributes - you can create your Vector object (even manually).

In this case you will get fast, memory efficient solution. A way faster than loading all into a XmlDocument.

Look here for example.

Charles Mager · Answer 3 · 2015-07-07T16:16:28.990

If you want to alleviate your issues in dealing with files this large, then stop loading the whole serialized XML into memory and then into the XML DOM when serializing and deserializing - I can't see that this is used for anything. XmlDocument will probably take at least 10x the amount of memory as the contents (which are also being held in memory), so you're likely using 100s of megabytes while trying to do it this way.

The following code should give the same results (though won't affect your file sizes), but you won't have the memory pressures that are causing your lockups and performance issues:

public void SerializeObject<T>(T serializableObject, string fileName)
{
    if (serializableObject == null) return;

    var serializer = new XmlSerializer(serializableObject.GetType());

    using (var stream = File.Open(fileName, FileMode.Create))
    {
        serializer.Serialize(stream, serializableObject);
    }
}

public T DeserializeObject<T>(string fileName)
{
    if (string.IsNullOrEmpty(fileName)) return default(T);

    var serializer = new XmlSerializer(typeof(T));

    using (var stream = File.Open(fileName, FileMode.Open))
    {            
        return (T) serializer.Deserialize(stream);
    }
}

Options for reducing file size? Compress the output (XML is very compressible) or change your serialization format.

XmlSerializer : Generated File too big

3 Answers3

What about binary serialization?