merge xml files in C#

Question

I'd like to merge a few xml files. The destination xml is slightly different then the source files. The destination file contains an aditional root element.

For example.

The destination xml:

<?xml version="1.0" encoding="utf-8"?>
<customer ID="A0001" name="customername">
.....
.....
</customer>

Source xml:

<?xml version="1.0" encoding="utf-8"?>
<order number="00001">
    <.....>
    <.....>
    <.....>
</order>

Every source xml file needs to be inserted between <customer ...> and </customer>

The source files can be very large (e.g. 2 Gb).

I can write the destination xml file with the root element and read the source files using XmlTextReader and

string myOrder = textReader.ReadOuterXml();
                        writer.WriteRaw(myOrder );

Result (where every order is a different xml file)

<?xml version="1.0" encoding="utf-8"?>
<customer ID="A0001" name="customername">
    <order number="00001">
        <.....>
        <.....>
        <.....>
    </order>
    <order number="00002">
        <.....>
        <.....>
        <.....>
    </order>
    <order number="00003">
        <.....>
        <.....>
        <.....>
    </order>
</customer>

But i'm afraid of out of memory exeptions for the large files using ReadOuterXml().

Any suggestion ?

Do you need to do *any* actual processing of the files, other than skipping the XML declaration? Is it just a case of adding a prefix and a suffix, and the rest is just copying? — Jon Skeet, Oct 08 '12 at 16:25
http://stackoverflow.com/questions/7671958/reading-large-xml-documents-in-net — Prix, Oct 08 '12 at 16:27
@Cuong Le: Yes sometimes even larger (can change that - out of my control) — John Doe, Oct 08 '12 at 16:30
@Jon Skeet: Just need to merge all the source files (and skipping the xml declaration) and add Thats all ;-) — John Doe, Oct 08 '12 at 16:32

Jon Skeet · Accepted Answer · 2012-10-08T16:46:18.493

It sounds like in this particular case, assuming all the files are really using UTF-8, you can basically cheat. .NET 4 makes this particularly easy:

public void MergeFiles(string outputPath, string prefix, string suffix,
                       IEnumerable<string> files)
{
    File.WriteAllText(outputPath, prefix);
    var lines = files.SelectMany(file => File.ReadLines(file).Skip(1));
    File.AppendAllLines(outputPath, lines);
    File.AppendAllText(outputPath, suffix);
}

This isn't quite as efficient as it might be, as it'll open the output file three times - but it's written about as simply as I could make it. Note that lines here is lazy - this won't read the source files completely into memory; it'll read a line at a time.

It does rely on each file starting with the XML declaration and being in UTF-8 though. There are far more robust streaming approaches you could use, but if you're confident of your source format, this is very simple...

EDIT: Sample usage:

string prefix = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n"
              + "<customer ID=\"A0001\" name=\"customername\">";
MergeFiles("output.xml", prefix, "</customer>", sourceFiles);

I'm confused, how does the customer tag get into the output? — Jodrell, Oct 08 '12 at 16:44
@Jodrell: You put that in the "prefix" part... I'll edit with an example. — Jon Skeet, Oct 08 '12 at 16:44
Ahh, I see now. I prefer the simplicity to the reader writer chaining I was preparing. — Jodrell, Oct 08 '12 at 17:00
@Jon Skeet: Sorry for my late response. I've been very busy. Nice solution. Works for me. — John Doe, Oct 12 '12 at 15:09

merge xml files in C#

1 Answers1