4

I have following code in code-behind of an ASP.Net app, where a file is being read followed by writing to the file.

Code

var state= File.ReadAllText(Server.MapPath(string.Format("~/state/{0}", fileName)));
if(state.indexOf("1") == 0) 
{
  File.WriteAllText(Server.MapPath(string.Format("~/state/{0}", fileName)), newState);
}

Sometimes, but not always, I get the following exception.

Exception

The process cannot access the file 'C:\inetpub\wwwroot\mywebsite1\state\20150905005929435_edf9267e-fad1-45a7-bfe2-0e6e643798b5' because it is being used by another process.

I am guessing that the file read operation sometimes is not closing the file before the write operation happens, Or may be the file write operation is not closing the file before the next request from web application comes. But, I cannot find what exactly is the reason.

Question: How can I avoid this error from happening? Is it not safe to use the File class and instead use the traditional approach of FileStream object where I would always dispose the FileStream object explicitly?

UPDATE 1

I tried a retry loop approach, but even that didn't seem to solve the problem , since I was able to reproduce the same error if the ASP.Net page was submitted very quickly multiple times one after another. So I am back to finding a fool-proof solution in my case.

  string state = null;
  int i = 0;
  while (i < 20) {
    try {

        state = File.ReadAllText(Server.MapPath(string.Format("~/state/{0}", fileName)));

    } catch (Exception ex2) {
        //log exception
        Elmah.ErrorSignal.FromCurrentContext().Raise(ex2);
        //if even retry doesn't work then throw an exception
        if (i == 19) {
            throw;
        }
        //sleep for a few milliseconds
        System.Threading.Thread.Sleep(10);
    }
    i++;
  }

  i = 0;
  while (i < 20) {
    try {


        File.WriteAllText(Server.MapPath(string.Format("~/state/{0}", fileName)), newState);

    } catch (Exception ex2) {
        //log exception
        Elmah.ErrorSignal.FromCurrentContext().Raise(ex2);
        //if even retry doesn't work then throw an exception
        if (i == 19) {
            throw;
        }
        //sleep for a few milliseconds
        System.Threading.Thread.Sleep(10);
    }
    i++;
  }

UPDATE 2

The only fool proof solution that seemed to work is by using a File Sequencing approach, as suggested by usr. This involves writing to a different file and not to the same file that was just read. The name of file being written to is the name of file that was just read appended by a sequence number.

  string fileName = hiddenField1.Value;
  string state = null;
  int i = 0;
  while (i < 20) {
    try {

        state = File.ReadAllText(Server.MapPath(string.Format("~/state/{0}", fileName)));

    } catch (Exception ex2) {
        //log exception
        Elmah.ErrorSignal.FromCurrentContext().Raise(ex2);
        //if even retry doesn't work then throw an exception
        if (i == 19) {
            throw;
        }
        //sleep for a few milliseconds
        System.Threading.Thread.Sleep(10);
    }
    i++;
  }

  i = 0;
  while (i < 20) {
    try {
        //***************FILE SEQUENCING**************************
        //Change the file to which state is written, so no concurrency errors happen 
        //between reading from and writing to same file. This is a fool-proof solution.
        //Since max value of integer is more than 2 billion i.e. 2,147,483,647
        //so we can be sure that our sequence will never run out of limits because an ASP.Net page
        //is not going to postback 2 billion times
        if (fileName.LastIndexOf("-seq_") >= 0) {
            fileName = fileName.Substring(0, fileName.LastIndexOf("-seq_") + 4 + 1) + (int.Parse(fileName.Substring(fileName.LastIndexOf("-seq_") + 4 + 1)) + 1);
        } else {
            fileName = fileName + "-seq_1";
        }
        //change the file name so in next read operation the new file is read
        hiddenField1.Value = fileName;
        File.WriteAllText(Server.MapPath(string.Format("~/state/{0}", fileName)), newState);

    } catch (Exception ex2) {
        //log exception
        Elmah.ErrorSignal.FromCurrentContext().Raise(ex2);
        //if even retry doesn't work then throw an exception
        if (i == 19) {
            throw;
        }
        //sleep for a few milliseconds
        System.Threading.Thread.Sleep(10);
    }
    i++;
  }

The only downside to above approach is that many files would get created as end users post back to the same ASP.Net page. So, it would be good to have a background job that deleted stale files so number of files would be minimized.

File Names with sequencing

File Sequencing Naming

UPDATE 3

Another fool proof solution is to alternate between read and write file names. This way we do not end up creating many files and only use 2 files as the end user posts back to the same page many times. The code is same as in code under UPDATE 2 except the code after FILE SEQUENCING comment should be replaced by code below.

if (fileName.LastIndexOf("-seq_1") >= 0) {
            fileName = fileName.Substring(0, fileName.LastIndexOf("-seq_1"));
        } else {
            fileName = fileName + "-seq_1";
        }

File Names with Alternating approach File Alternating Approach

Sunil
  • 20,653
  • 28
  • 112
  • 197
  • 1
    This question is slightly misguided. The static methods on the `File` type are just convenient wrappers around `using (FileStream)`, so them being static is entirely irrelevant. Your problem is with concurrency, i.e. two processes trying to act on the same file at the same time. How to circumvent such issues is handled in many other questions, try searching. – CodeCaster Sep 05 '15 at 13:37
  • 1
    Related: [How to handle concurrent file access with a filestream/streamwriter?](http://stackoverflow.com/questions/1882637/how-to-handle-concurrent-file-access-with-a-filestream-streamwriter), [Handling concurrent file writes](http://stackoverflow.com/questions/4551640/handling-concurrent-file-writes), [Raised exception in the case of concurrent file access with StreamReader](http://stackoverflow.com/questions/4324719/raised-exception-in-the-case-of-concurrent-file-access-with-streamreader), ... – CodeCaster Sep 05 '15 at 13:41
  • @CodeCaster, This problem is happening on my dev machine. I am the only person running the ASP.Net app from Visual Studio 2013 on my laptop. So why should concurrency happen? – Sunil Sep 05 '15 at 14:01
  • 2
    I'd disable any anti-malware that is running on the machine and see if the problem goes away. I've had anecdotal experience that some AV's will keep a file open longer than the actual user of the file. – Michael Burr Sep 07 '15 at 08:18
  • @MichaelBurr, That's an excellent point. I read somewhere that anti-virus software can result in file being opened longer than expected. I think this concurrency problem needs a defensive approach of programming, like re-try looping or file sequencing and better if both approaches are used. So the aim is to minimize concurrency errors and not to 100% overcome them. – Sunil Sep 07 '15 at 22:42

2 Answers2

5

I am guessing that the file read operation sometimes is not closing the file before the write operation happens, Or may be the file write operation is not closing the file before the next request from web application comes.

Correct. File systems do not support atomic updates well. (Especially not on Windows; many quirks.)

Using FileStream does not help. You would just rewrite the same code that the File class has. File has no magic inside. It just uses FileStream wrapped for your convenience.

Try keeping files immutable. When you want to write a new contents write a new file. Append a sequence number to the file name (e.g. ToString("D9")). When reading pick the file with the highest sequence number.

Or, just add a retry loop with a small delay.

Or, use a better data store such as a database. File systems are really nasty. This is an easy problem to solve with SQL Server for example.

usr
  • 168,620
  • 35
  • 240
  • 369
  • I will try a retry loop with a Thread.Sleep(10) and loop counter <20. I think with file i/o things are never guaranteed. May be I should use something like SQLite database, which is similar to file i/o. – Sunil Sep 05 '15 at 14:36
  • I tried using retry loop as in code under UPDATE 1, and was still able to reproduce the same error if I submitted the ASP.Net page very quickly multiple times. My Asp.Net page gets submitted if I click on a page number in a grid pager. So it seems that even retry loop is not really a fool-proof solution. – Sunil Sep 05 '15 at 15:05
  • If that loop doesn't work then something is keeping the file permanently open. Find what it is and kill it. – usr Sep 05 '15 at 15:18
  • Or may be increase the sleep interval and also the max number of loops, Max number of loop is 20 and sleep interval is 10 milliseconds in my code. – Sunil Sep 05 '15 at 15:23
  • The solutions under UPDATE 2 and UPDATE 3 of my original post work without any concurrency errors and they seem fool proof. Thanks for your ideas on this. – Sunil Sep 05 '15 at 17:29
  • You *can* have atomic file modifications using Transactional NTFS. Unfortunately, the functionality isn't exposed by System.IO, you'd need a library like AlphaFS to do this. – Panagiotis Kanavos Sep 07 '15 at 09:57
  • @PanagiotisKanavos that is true from an atomicity standpoint. But concurrent "transactions" are not isolated. They might abort instead of observing a serializable state history. For example you can't rename on NTFS instantly. There is always a window of time where the file being renamed is locked. I found that out the hard way. – usr Sep 07 '15 at 10:10
0

I am guessing that the file read operation sometimes is not closing the file before the write operation happens

Although according to the documentation the file handle is guaranteed to be closed by this method, even if exceptions are raised, the timing of the closing is not guaranteed to happen before the method returns: the closing could be done asynchronously.

One way to fix this problem is to write the results into a temporary file in the same directory, and then move the new file in place of the old one.

Sergey Kalinichenko
  • 714,442
  • 84
  • 1,110
  • 1,523
  • 1
    Is this documented somewhere ? This has rather huge implication if the system doesn't allow concurrently opening a file (As is rather common on windows) – nos Sep 07 '15 at 08:00
  • 1
    It's unreasonable to read that as meaning that the close can happen asynchronously. Following this logic, you could say something similar about nearly any method that had a side effect not directly observable in the return value. – Michael Burr Sep 07 '15 at 08:56
  • @MichaelBurr Why? Unless a method is called for its side effect, and when the side effect is not directly observable in return value, it is entirely reasonable to let method designers complete the side effects asynchronously. – Sergey Kalinichenko Sep 07 '15 at 12:09
  • @dasblinkenlight Please also read [Is “sequential” file I/O with System.IO.File helper methods safe?](http://stackoverflow.com/q/32433630/1207195), it's about this answer. It's _reasonable_ a side-effect ends asynchronously but (IMO) in disk I/O to close handle is too strongly related to _main_ task for a method required to be _atomic_ (from library POV). – Adriano Repetti Sep 07 '15 at 14:44
  • 2
    @dasblinkenlight: Aside from this language-laywer discussion about whether the close could occur asynchronously, the reference implementation disposes of the stream object used (which closes the file handle) before returning. I'm sure the actual implementation does as well - it would be more work and more complex to do otherwise. Not to mention surprising nearly every user of the API. – Michael Burr Sep 07 '15 at 18:07