Unit Test Xml encoding issue

Question

Consider following class:

public class SampleXmlGenerator
{
    public byte[] GenerateDocumentBytes()
    {
        byte[] fileBytes;
        using (var xmlStream = new MemoryStream())
        {
            using (var myWriter = new XmlTextWriter(xmlStream, Encoding.GetEncoding("UTF-8")))
            {
                myWriter.Formatting = Formatting.Indented;
                myWriter.Indentation = 4;
                myWriter.IndentChar = ' ';
                myWriter.WriteStartDocument();
                myWriter.WriteStartElement("foo");
                myWriter.WriteString("bar");
                myWriter.WriteEndElement(); // end foo

                myWriter.Flush();

                fileBytes = xmlStream.ToArray();
            }
        }

        return fileBytes;
    }
}

With following Unit Test:

[TestClass]
public class TestSampleXmlGenerator
{
    [TestMethod]
    public void TextEmptyDocument()
    {
        var actualBytes = new SampleXmlGenerator().GenerateDocumentBytes();
        var actualUtf8String = Encoding.UTF8.GetString(actualBytes);
        Console.Out.WriteLine("// actualUtf8String");
        Console.Out.WriteLine(actualUtf8String);

        var actualDefaultString = Encoding.Default.GetString(actualBytes);
        Console.Out.WriteLine("// actualDefaultString");
        Console.Out.WriteLine(actualDefaultString);


        var expectedString = @"<?xml version=""1.0"" encoding=""utf-8""?>
<foo>bar</foo>";
        var expectedBytes = Encoding.UTF8.GetBytes(expectedString);

//      var expectedBytes = Encoding.Convert(Encoding.Default, Encoding.UTF8, Encoding.Default.GetBytes(@"<?xml version=""1.0"" encoding=""utf-8""?>
//<foo>bar</foo>"));
//      var expectedString = Encoding.UTF8.GetString(expectedBytes);

        Console.Out.WriteLine("// expectedString");
        Console.Out.WriteLine(expectedString);

        Assert.AreEqual(expectedBytes.Length, actualBytes.Length);
        //Assert.AreEqual(expectedString, actualUtf8String);
    }
}

And finally output:

Assert.AreEqual failed. Expected:<54>. Actual:<57>. 

// actualUtf8String
<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>

// actualDefaultString
ï»¿<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>

// expectedString
<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>

expectedString and actualUtf8String look the same, but are not.

actualDefaultString shows the 3 extra chars at the beginning.

So what gives? How do I go about testing/comparing generated XML? What should I do differently?

There is a constructor https://msdn.microsoft.com/en-us/library/s064f8w2(v=vs.110).aspx to eliminate the byte order mark. — Martin Honnen, Sep 21 '16 at 20:53
Thanks Martin, but I did not want to eliminate BOM from document, just during testing ignore it. But your link did point me in the right direction, and what to look for. — CrnaStena, Sep 22 '16 at 14:02
why not make a Test function that checks the BOM and returns the rest as a string? — Keith Nicholas, Sep 22 '16 at 21:23

score 1 · Accepted Answer · answered Sep 21 '16 at 21:57

1

what you are seeing is BOM preamble

https://msdn.microsoft.com/en-us/library/system.text.encoding.getpreamble(v=vs.110).aspx

you can either test for it, or not serialize it.

answered Sep 21 '16 at 21:57

Keith Nicholas

43,549
15
93
156

I ended up removing in the Unit Test, as I did not want to change Xml generation process. – CrnaStena Sep 22 '16 at 14:03

score 0 · Answer 2 · edited May 23 '17 at 10:27

Based on suggestions from Martin and Keith, and with some additional research I ended up removing BOM from generated XML bytes in Unit Test, in following manner (based on following SO article):

        var xmlBytes = new SampleXmlGenerator().GenerateDocumentBytes();
        var newXmlDoc = new XmlDocument {PreserveWhitespace = true};
        newXmlDoc.Load(new MemoryStream(xmlBytes));
        var actualBytes = Encoding.UTF8.GetBytes(newXmlDoc.OuterXml);

Now Unit Test passes!

Unit Test Xml encoding issue

2 Answers2