14

I have an application that must check a folder and read any files that are copied into it. How do I test if a file in that folder is currently being written to? I only want to read files that have had all their data written to them and are closed.

willsan
  • 141
  • 1
  • 1
  • 3

9 Answers9

17

It ain't clean, but it works:

try
{
    using (Stream stream = new FileStream("File.txt"))
    {
    }
} 
catch 
{
    // Check if file is in use (or whatever else went wrong), and do any user communication that you need to
}

I don't particularly like using exceptions as conditions, but as far as I know this is about the easiest and best (and only reliable) way to do this.

Kyle Rosendo
  • 25,001
  • 7
  • 80
  • 118
  • I think try-catch is basically the only fail-safe way to test the file. Too bad they don't have a IsOpen property in the File or FileInfo classes.. – KP. Mar 25 '10 at 13:22
  • 1
    +1: This is the **ONLY** way to do it. Imaging there's a call on the file object "IsOpen", there is nothing stopping another process locking the file between you calling IsOpen and you opening the file. The only way to see if you can get a lock is to try to lock it (and then keep it locked). – Binary Worrier Mar 25 '10 at 13:24
  • The problem he's trying to solve is inherently prone to race conditions. As I explained in my answer below, actually the approach listed above will fail if the "writing process" writes one chunk at a time (closing the file between consecutive writes). I think having such a process is much more likely than having a process that wrote nothing for X seconds, and then suddenly restarts writing just as you start doing your job. Anyway, regardless of the approach, additional syncrhonization is needed, for sure. – Virgil Mar 25 '10 at 13:29
  • @Binary - Added the "and only reliable", thanks! @Virgil - That is true, but fact is that there's no way to lock a file (or see if its locked) for use, without trying to lock it yourself. – Kyle Rosendo Mar 25 '10 at 13:29
  • Kyle, I disagree, process explorer does just that. You could obviously do it if you put in a lot of effort, and use pinvoke. For example, you could list all the processes that run currently, and then find all the handles that are opened by those processes. – Virgil Mar 25 '10 at 13:41
  • 1
    @Virgil - and then as your method finds no handles and is about to return, the file gets locked by another process. Then what? – Kyle Rosendo Mar 25 '10 at 13:57
  • +1: I agree this is the best solution. Put this in a loop and create a maximum attempt so the app doesn't crash due to some external program never releasing a filelock (maybe because it crashed as well). – galford13x Mar 25 '10 at 14:14
  • As said, the problem itself is inherently prone to race conditions. The answer to your "then what" question is simply "then you find it later again as being modified". Whatever you would do, from the moment you get your "file list" until you use it, someone else may write in a new/existing file there. That is - unless you lock out the entire folder, but that's typically not very smart :) – Virgil Mar 25 '10 at 14:54
  • @Virgil: Could you expand on your "is simply" comment above? As far as I can see your hypothetical "IsFileOpen" check is redundant, you will always need to have the exception handler in place regardless of the speed/efficiency/safety of the "IsFileOpen" check. Why not just rely on the exception handler? – Binary Worrier Mar 25 '10 at 15:41
  • Well, let's start again with the problem statement: "I only want to read files that have had all their data written to them and are closed". There are two requirements here: 1. are closed 2. have had all the data written to them My claim is that you can find all the files that were open at a given moment in time ( and simply assume that the rest were closed). I.e, for point 1, you don't *NEED* to resort to locking the files yourself. Of course, solving point 1 doesn't guarantee point 2 - i.e. if a file was closed when you looked at it, there's no guarantee that someone won't open it later. – Virgil Mar 26 '10 at 07:47
  • That being said, return to requirement2: "have all data written to them". Is there any way to be absolutely sure of that, without knowing the writer process? No! So any attempt is futile - of course you can use the locking, but it will just (temporarily) prevent the writer process from writing additional data - it will NOT guarantee that all data is written. In that respect, assuming that you don't know the "writer processes" - checking the timestamps is a likely a better heuristic than locking the file. Neither of them is guaranteed to solve your problem, you still need a failsafe mechanism. – Virgil Mar 26 '10 at 07:52
  • That failsafe is what I meant with my "is simply" answer - whatever you do, you need to re-inspect the file for later modification. [sorry, I needed three comments to answer due to message length limitations] – Virgil Mar 26 '10 at 07:54
2

Perhaps opening the file exclusively like this -

System.IO.File.Open(PathToFile, FileMode.Open, FileAccess.ReadWrite, FileShare.None);

That could be placed in a loop with a try/catch until its successful.

kragan
  • 989
  • 5
  • 8
  • 1
    You may want a conditional retry in the catch. I typically have a counter and a Thread.Sleep(500) so I don't pound the I/O and can still bail if something such as a disk crash occures. – Matthew Whited Mar 25 '10 at 15:03
2

Catching exception is expensive you should try to use this:

http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx

There's a super clear example on the msdn page. That'll give you nice events on change, created, delete, renamed.

copy the file into the folder and rename when your done catch the rename event and your sure your not reading files that aren't finished yet.

Or you could catch the change events until (this is tricky) they aren't fired anymore.

albertjan
  • 7,739
  • 6
  • 44
  • 74
  • Unfortunately, in my case the file being written to is from a third party program. – Ernest Mar 04 '14 at 23:09
  • @ErnestSoeralaya That shoudn't matter this works for all programs writing to the file you're watching. – albertjan Mar 05 '14 at 07:37
  • Wouldn't renaming the file being generated require the ability to change the behavior of such program? – Ernest Mar 05 '14 at 18:51
  • 1
    A FSW could serve as a launching point for determining whether some file has changed, however the change notification doesn't come with any guarantees: it's just an event saying that a file has been written to (for example). It does not tell you whether 1) the write is completed, 2) whether the file still exists or 3) whether there are any other locks on the file. – rianjs May 23 '14 at 17:02
1

The only thing I can think of is that a file lock will be put on the file while it is being written to, therefore preventing you from writing to it (throwing a System.IO exception of file being locked by another process), but this is not foolproof. For example, the file could be written to in chunks by opening and closing the file for each chunk.

So you could do this with a try/catch block:

try
{
   //opening the file
} 
catch(IOException ex)
{
   // file is open by another process
}
Bryan Denny
  • 27,363
  • 32
  • 109
  • 125
1

When you open it with System.IO.File.OpenWrite(filename), it should throw IOException.

So, in a try catch, the code being executed in the catch block should tell you the file was open.

Kyle Rosendo
  • 25,001
  • 7
  • 80
  • 118
David Fox
  • 10,603
  • 9
  • 50
  • 80
1

The file that is written to will typically have write locks on it so if you try to open it for write, you'll fail; but OTOH you may have a process that opens, writes, and closes the file - then repeats. I think the simplest and safest way would be simply to check for files that were written to recently, but not TOO recent (e.g. not in the last 2 seconds or so)

[l.e.] Another point to consider is that if you have a process that writes data in chunck (open-write-close), and you get an exclusive lock for the file (to check that the file is no longer used) - than your external process will fail. I think it's far better to check the timestamps, and only get the exclusive lock after you're reasonably sure that the other process finished writing.

Virgil
  • 3,022
  • 2
  • 19
  • 36
  • The external process should handle such the case as well. To give an example as your talking about, writing to a log, or common log, shared between processes. The log is opened whenever it is needed by some process, that process is responsible for gaining the file lock safely. If it doesn't gain the file lock is should handle that exception properly such as sleeping for a bit or trying again at some later time. If your just looking at time stamps then there is no reliable way to know by the timestamp that the log will be written to when you try to get the file lock. – galford13x Mar 25 '10 at 14:08
  • The keyword here is "should". We all now that not all applications are .... perfect. The idea is that you know few things about the external process that writes those files, and that process is probably well outside your control. – Virgil Mar 25 '10 at 14:48
0

Number of options. The one that springs to mind straight away is to try opening the file for write. If you raise an exception, it is still being written to. Wait for a defined period and try again.

Alternatively check the timestamps and only try reading the file 1 or 2 minutes after the timestamp last changed

CResults
  • 5,100
  • 1
  • 22
  • 28
0

Use the System.IO.FileSystemWatcher to determine files that are changed.

Open them in an exclusive reader (FileMode.Read) in make the copies, and catch the exception should this fail (because someone else has it open for writing).

Will
  • 73,905
  • 40
  • 169
  • 246
0

If you have any control over the process that does the writing in, then consider augmenting it so the following process happens:

  1. Process starts writing file "filename.xyz"
  2. Process writes more of file "filename.xyz"
  3. ....
  4. ....
  5. Process finishes writing file "filename.xyz"
  6. Process writes another (zero-length, empty) file to the folder called "filename.xyz.done"

You can look for ".done" files as the trigger to start processing a file. Again, this will only work if you have any way to modify the process that does the writing in of files.

Rob
  • 45,296
  • 24
  • 122
  • 150