How do I append to a file in an Azure storage file share?

Question

I want to write entries to a log file stored in Azure file storage. I currently have this:

var log = "My log entry";

var client = _storageAccount.CreateCloudFileClient();
var share = client.GetShareReference(Config.LogShare);
share.CreateIfNotExists();
var root = share.GetRootDirectoryReference();
var logfile = root.GetFileReference("log.txt");
if (!logfile.Exists()) logfile.Create(0);

// What goes here to append to the file...?

I can see plenty of examples of how to do this with Blobs, or how to upload an entire file, but how do I just append to an existing file?

I have tried this:

var buffer = Encoding.GetEncoding("UTF-8").GetBytes(log.ToCharArray());

using (var fileStream = logfile.OpenWrite(0)) {
    fileStream.Write(buffer, (int)logfile.Properties.Length, buffer.Length);
}

But then I get this error:

The remote server returned an error: (416) The range specified is invalid for the current size of the resource..

score 5 · Answer 1 · answered May 19 '16 at 08:25

I managed to work this out myself. You just need to increase the size of the file by the number of new bytes you want to write to it, and then write the new data to that new empty space at the end of the file, like this:

var client = _storageAccount.CreateCloudFileClient();
var share = client.GetShareReference(Config.LogShare);
share.CreateIfNotExists();
var root = share.GetRootDirectoryReference();
var logfile = root.GetFileReference("log.txt");
if (!logfile.Exists()) logfile.Create(0);

var buffer = Encoding.UTF8.GetBytes($"{log}\r\n");

logfile.Resize(logfile.Properties.Length + buffer.Length);

using (var fileStream = logfile.OpenWrite(null)) {
    fileStream.Seek(buffer.Length * -1, SeekOrigin.End);
    fileStream.Write(buffer, 0, buffer.Length);
}

Yeah, that's something like "append", but it's actually done by 1. resize, 2. write data to the new area. Append should be an atomic operation which directly append data and make file size larger. :) — Zhaoxing Lu, May 19 '16 at 09:45
fileStream.Seek(buffer.Length * -1, SeekOrigin.End); This line produces an error. "Azure.Storage.Files.Shares: Specified method is not supported." — Chamika Goonetilaka, May 22 '21 at 09:10

score 1 · Answer 2 · edited Nov 20 '19 at 09:31

1

Azure file storage REST API doesn't support appending to an existing file. To achieve this, please mount the file share to your machine as a drive, and append to the file just like simple local files.

Actually, I don't think you really need appending functionality per your code above. You can specify the file size in CloudFile.OpenWrite() / CloudFile.Create(), or try CloudFile.UploadFromStream() instead of CloudFile.OpenWrite().

edited Nov 20 '19 at 09:31

Paul Chibulcuteanu

184
1
9

answered May 18 '16 at 22:47

Zhaoxing Lu

6,319
18
41

Actually you can append... see my answer :) – BG100 May 19 '16 at 08:26

score 1 · Answer 3 · answered Sep 01 '17 at 22:01

1

You can do this with blobs https://blogs.msdn.microsoft.com/windowsazurestorage/2015/04/13/introducing-azure-storage-append-blob/

Shame it doesn't work with files too

answered Sep 01 '17 at 22:01

Simon_Weaver

140,023
84
646
689

1

Looks like the original blog link is now redirecting to the generic Azure blog page, but here is a [blog post for the greater Azure Storage that mentions "append blob"](https://azure.microsoft.com/blog/azure-storage-release-append-blob-new-azure-file-service-features-and-client-side-encryption-ga/) and a [docs page with some details](https://learn.microsoft.com/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-append-blobs). – patridge Jun 09 '20 at 18:12

Assil · Answer 4 · 2017-07-10T00:01:00.833

This error could also be due to multi-threaded access. I bet if you tried to lock the file before you access it, you will not face this problem. There are many ways to update the file. Since you already managed to get the share, the root, the folder and the file.. Here is a portion of my code that worked for me.

if (!fileLock.IsWriteLockHeld) fileLock.EnterWriteLock();


        try
        {
            using (var stream = new MemoryStream(content, false))
            {
                file.UploadFromStream(stream, null, options);
            }
        }
        catch (Exception ex)
        {
            File.AppendAllText(FileName, ex.ToString());
        }
        finally
        {
            if (fileLock.IsWriteLockHeld)
                fileLock.ExitWriteLock();
        }

Where fileLock is declared as:

  protected ReaderWriterLockSlim fileLock = new ReaderWriterLockSlim();

Having said that, I am not saying that this is the best way ever to do it. The two things I would like you to keep in mind : 1-Lock the resource that is likely to be accessed by more than one thread (That is so common in AZURE) 2- Get familiar with asynchronous methods that Azure provides.. use them when they suit well.

Coming back to your original problem about appending to the existing file.. All the methods of the CloudFile will overwrite the existing file. Cloud Files are not for frequent writing, and they indeed impact performance if you keep writing on them frequently, add the lock impact on performance, they will be horrible.

Cloud files are meant to store big bulk of data once and for all, if you want to add another bulk you have the choice of creating another file. Have all your data with the client till they reach some size and create an algorith to select the file name and upload them all at once.

How do I append to a file in an Azure storage file share?

4 Answers4

Linked