Does File.AppendAllText manage collisions (i.e. multi-user concurrency)?

Question

Question

Does File.AppendAllText manage collisions from multiple writers?

Research

I noticed that the MSDN documentation doesn't really provide a position either way, so I decided I'd reflect the code and just see what it does. Below is the called method from File.AppendAllText:

private static void InternalAppendAllText(string path, string contents, Encoding encoding)
{
    using (StreamWriter streamWriter = new StreamWriter(path, true, encoding))
    {
        streamWriter.Write(contents);
    }
}

and as you can see it simply leverages a StreamWriter. So, if we dig a little deeper into that, specifically the constructor it uses, we find that it ultimately calls this constructor:

internal StreamWriter(string path, bool append, Encoding encoding, int bufferSize, bool checkHost) : base(null)
{
    if (path == null)
    {
        throw new ArgumentNullException("path");
    }
    if (encoding == null)
    {
        throw new ArgumentNullException("encoding");
    }
    if (path.Length == 0)
    {
        throw new ArgumentException(Environment.GetResourceString("Argument_EmptyPath"));
    }
    if (bufferSize <= 0)
    {
        throw new ArgumentOutOfRangeException("bufferSize", Environment.GetResourceString("ArgumentOutOfRange_NeedPosNum"));
    }
    Stream streamArg = StreamWriter.CreateFile(path, append, checkHost);
    this.Init(streamArg, encoding, bufferSize, false);
}

with the following values:

path:        the path to the file
append:      the text to append
encoding:    UTF8NoBOM
bufferSize:  1024
checkHost:   true

and further we find that the base(null) implementation doesn't really do anything but set the InternalFormatProvider to null. So, if we keep digging we find that CreateFile:

private static Stream CreateFile(string path, bool append, bool checkHost)
{
    FileMode mode = append ? FileMode.Append : FileMode.Create;
    return new FileStream(path, mode, FileAccess.Write, FileShare.Read, 4096, FileOptions.SequentialScan, Path.GetFileName(path), false, false, checkHost);
}

creates a FileStream with these parameter values:

path:         the path to the file
mode:         FileMode.Append
access:       FileAccess.Write
share:        FileShare.Read
bufferSize:   4096
options:      FileOptions.SequentialScan
msgPath:      just the file name of the path provided
bFromProxy:   false
useLongPath:  false
checkHost:    true

an so now we're finally getting somewhere because we're about to leverage the Windows API, and this is where the question really begins because that FileStream::ctor calls a method named Init. It's a pretty long method, but I'm really interested in one line:

this._handle = Win32Native.SafeCreateFile(text3,
    dwDesiredAccess,
    share,
    secAttrs,
    mode,
    num,
    IntPtr.Zero);

which of course calls CreateFile, where the parameter values are:

text3:            the full path to the file
dwDesiredAccess:  1073741824
share:            1 (FILE_SHARE_READ)
secAttrs:         null
mode:             4 (OPEN_ALWAYS)
num:              134217728 | 1048576 (FILE_FLAG_SEQUENTIAL_SCAN | FILE_FLAG_POSIX_SEMANTICS)

so, what would Windows do if I had two threads trying to access that call at the same time for the same path? Would it open the file and buffer the writes so that both consumers are allowed to write to the file? Or do I need to leverage a lock object and lock around the call to AppendAllText?

Answer 1

Only one will win for writing, and it will be the first, any subsequent attempts will fail until the write lock is released (i.e. the buffer is flushed and the file closed) - however, it could be open for reading simultaneously (permissions depending).

Read - Allows subsequent opening of the file for reading. If this flag is not specified, any request to open the file for reading (by this process or another process) will fail until the file is closed. However, even if this flag is specified, additional permissions might still be needed to access the file.

Answer 2

The key is this method:

private static Stream CreateFile(string path, bool append, bool checkHost)
{
    FileMode mode = append ? FileMode.Append : FileMode.Create;
    return new FileStream(path, mode, FileAccess.Write, FileShare.Read, 4096, FileOptions.SequentialScan, Path.GetFileName(path), false, false, checkHost);
}

It's opening with FileShare.Read, meaning that other threads or processes can open the file for reading, but no other process/thread can open it for writing.

You probably wouldn't want it to allow multiple concurrent writers. Consider writing two very large buffers. It's very likely that they would end up being interleaved.

So, yes ... if you have multiple threads that might be appending to that file, you need to synchronize access, probably with a lock.

Another option, depending on your application, would be to have a consumer thread that reads text from a queue and appends to the file. That way, only one thread has access to the file. Other threads put the messages on a queue that the writer thread services. This is pretty easy to do with BlockingCollection, but is probably overkill unless you're writing to the file on a continual basis (as in logging).

Answer 3

I knew the topic is old, but I found out that after read https://stackoverflow.com/a/18692934/3789481

By implement EventWaitHandle, I can easily prevent collision with File.AppendAllText

    EventWaitHandle waitHandle = new EventWaitHandle(true, EventResetMode.AutoReset, "SHARED_BY_ALL_PROCESSES");

    Task[] tasks = new Task[ThreadCount];
    for (int counter = 0; counter < ThreadCount; counter++)
    {
        var dividedList = ....
        tasks[counter] = await Task.Factory.StartNew(async () => await RunTask(counter, dividedList, waitHandle));
    }

And the RunTask write to file

    private static async Task RunTask(int threadNum, List<string> ids, EventWaitHandle waitHandle)
    {
        Console.WriteLine($"Start thread {threadNum}");

        foreach (var id in ids)
        {
            // start waiting
            waitHandle.WaitOne();
            File.AppendAllText(@".\Result.txt", text + Environment.NewLine);
            waitHandle.Set();
            // until release

            // ninja code
        }
        Console.WriteLine($"End thread {threadNum}");
    }

I have tested with 500 threads, it worked well !!