Why can't I append a new byte array to a file using FileStream?

Question

I am trying to write a byte array to a file, multiple times. The FileMode is set to Append, to create the file if it doesn't exist, otherwise open the file and seek to the end of the file as described. The problem is that when writing to the existing file, it gets overwritten rather than have the new byte array appended to it. That's all there is to it.

void WriteToFile()
{
    byte[] buffer = new byte[16 * 1024];
    int num;

    using (FileStream dest_stream = new FileStream(filename, FileMode.Append, FileAccess.Write))
    {
        while (num = ReadFromAnotherStream(my_other_stream, ref buffer) > 0)
            dest_stream.Write(buffer, 0, num);
    }
}

This function will be called occasionally. If the file already exists, seek to the end of the file and continue writing from there, otherwise create a new file and write data.

---

When it should append, it overwrites... It does not append.

It should append to the file instead of overwrite it.

There is no error thrown.

Using Seek for the FileStream does nothing.

When it overwrites, the data is correct, however, it needs to be appended at the end of the previous data, and not overwrite.

---

UPDATE: Well, I had no choice but to divide each call into multiple "temp" files then at the end merge them into the main file. Such worked flawlessly, no Seeking was required leading to non-corrupted files.

A downside would be extra processing for multiple temp files (especially large ones) being merged into one.

Pseudo-code:

string filename;
List<string> tmp_files = new List<string>();
int __i = 0;
do
{
    filename = $"my_file.tmp{__i++}";
    tmp_files.Add(filename);
}
while (File.Exists(filename));

// Every time WriteToFile() gets called... This should be on top.
// Instead of writing directly to the file, add more of them until the input stream has been read fully.
using (FileStream out_fs = new FileStream("my_file.bin", FileMode.Create))
{
    foreach (string tmp_file in tmp_files)
    {
        using (FileStream in_fs = new FileStream(tmp_file, FileMode.Open))
        {
            in_fs.CopyTo(out_fs);
        }
        File.Delete(tmp_file);
    }
}

First of all, thank you everyone who took part in this thread. Part of the code could not be shared and that is beyond me. I understand that there is no magic ball out there to read minds from pseudo-codes, but I was in desperate to solve this unethical mistake from the API, so I wanted to get as many possibilities.

I still don't know what the issue is with Append, but the input stream has absolutely nothing to do with it and that's out of the way for now.

Answer 1

We can't tell what is wrong with your code because you didn't include the code for ReadFromAnotherStream. But here is some code that does what you want, and it works:

    /// <summary>
    /// Appends the contents of the file at inputFilePath to the file at pathToAppendTo
    /// </summary>
    void Append(string inputFilePath,string pathToAppendTo)
    {

        var buffer = new byte[16];

        using FileStream source = new FileStream(inputFilePath, FileMode.Open, FileAccess.Read);
        using FileStream destinationStream = new FileStream(pathToAppendTo, FileMode.Append, FileAccess.Write);
        while (TryRead(source, buffer, out int bytes))
        {
            destinationStream.Write(buffer, 0, bytes);
        }
    }
    
    private bool TryRead(FileStream source, byte[] buffer, out int bytesRead)
    {
        bytesRead = source.Read(buffer, 0, buffer.Length);
        return bytesRead > 0;
    }

Here is a unit test to verify that it works:

    [TestMethod]
    public void TestWriteFile()
    {
        var inputFileName = "C:/fileYouWantToCopy";
        var outputFileName = "C:/fileYouWantToAppendTo";

        var originalFileBytes = GetFileLength(outputFileName);
        var additionalBytes = GetFileLength(inputFileName);

        Append(inputFileName,outputFileName);
        Assert.AreEqual(GetFileLength(outputFileName), originalFileBytes + additionalBytes);
        
    }

    private long GetFileLength(string path) => new FileInfo(path).Length;