10

I want to use FileSystemWatcher to monitor a directory and its subdirectories for files that are moved. And then I want to trigger some code when all the files have been moved. But I don't know how. My code as is will trigger each time a file is moved, and if a user moves several files at once I only want it to trigger once for all files. So basically I want to create a list, and once the moving of all files is done I want to do stuff to that list...

Here's the code:

class Monitor
{
    private List<string> _filePaths;  
    public void CreateWatcher(string path)
    {
        FileSystemWatcher watcher = new FileSystemWatcher();

        watcher.Filter = "*.*";

        watcher.Created += new
        FileSystemEventHandler(watcher_FileCreated);

        watcher.Path = path;
        watcher.IncludeSubdirectories = true;

        watcher.EnableRaisingEvents = true;
    }

    void watcher_FileCreated(object sender, FileSystemEventArgs e)
    {
        _filePaths.Add(e.FullPath);
        Console.WriteLine("Files have been created or moved!");
    }

}

UPDATE: Trying to use Chris's code, but it doesn't work (see my comment at Chris's answer):

class Monitor
    {
        private List<string> _filePaths;
        private Timer _notificationTimer;
        private FileSystemWatcher _fsw;
        public Monitor(string path)
        {
            _notificationTimer = new Timer();
            _notificationTimer.Elapsed += notificationTimer_Elapsed;
            // CooldownSeconds is the number of seconds the Timer is 'extended' each time a file is added.
            // I found it convenient to put this value in an app config file.
            int CooldownSeconds = 1;
            _notificationTimer.Interval = CooldownSeconds * 1000;

            _fsw = new FileSystemWatcher();
            _fsw.Path = path;
            _fsw.IncludeSubdirectories = true;
            _fsw.EnableRaisingEvents = true;

            // Set up the particulars of your FileSystemWatcher.
            _fsw.Created += fsw_Created;
        }

        private void notificationTimer_Elapsed(object sender, ElapsedEventArgs e)
        {
            //
            // Do what you want to do with your List of files.
            //
            Console.Write("Done");
            // Stop the timer and wait for the next batch of files.            
            _notificationTimer.Stop();
            // Clear your file List.
            _filePaths = new List<string>();
        }


        // Fires when a file is created.
        private void fsw_Created(object sender, FileSystemEventArgs e)
        {
            // Add to our List of files.
            _filePaths.Add(e.Name);

            // 'Reset' timer.
            _notificationTimer.Stop();
            _notificationTimer.Start();
        }


    }

UPDATE 2:

Tried this according to Anders's answer:

public class FileListEventArgs : EventArgs
{
    public List<string> FileList { get; set; }
}

public class Monitor
{
    private List<string> filePaths;
    private ReaderWriterLockSlim rwlock;
    private Timer processTimer;
    public event EventHandler FileListCreated;


    public void OnFileListCreated(FileListEventArgs e)
    {
        if (FileListCreated != null)
            FileListCreated(this, e);
    }

    public Monitor(string path)
    {
        filePaths = new List<string>();

        rwlock = new ReaderWriterLockSlim();

        FileSystemWatcher watcher = new FileSystemWatcher();
        watcher.Filter = "*.*";
        watcher.Created += watcher_FileCreated;

        watcher.Path = path;
        watcher.IncludeSubdirectories = true;
        watcher.EnableRaisingEvents = true;
    }

    private void ProcessQueue()
    {
        List<string> list = new List<string>();
        try
        {
            Console.WriteLine("Processing queue, " + filePaths.Count + " files created:");
            rwlock.EnterReadLock();

        }
        finally
        {
            if (processTimer != null)
            {
                processTimer.Stop();
                processTimer.Dispose();
                processTimer = null;
                OnFileListCreated(new FileListEventArgs { FileList = filePaths });
                filePaths.Clear();
            }
            rwlock.ExitReadLock();
        }
    }

    void watcher_FileCreated(object sender, FileSystemEventArgs e)
    {
        try
        {
            rwlock.EnterWriteLock();
            filePaths.Add(e.FullPath);

            if (processTimer == null)
            {
                // First file, start timer.
                processTimer = new Timer(2000);
                processTimer.Elapsed += (o, ee) => ProcessQueue();
                processTimer.Start();
            }
            else
            {
                // Subsequent file, reset timer. 
                processTimer.Stop();
                processTimer.Start();
            }

        }
        finally
        {
            rwlock.ExitWriteLock();
        }
    }

I had to move the event trigger into the finally statement, and that works. I don't know if there is some reason I wouldn't want to do that?

Anders
  • 12,556
  • 24
  • 104
  • 151
  • Will you get events from the directory where the file is being removed/added, as well as for the moved file(s)? If so, perhaps you could just ignore (or filter) the file events and only listen to the directory events? Edit: you probably get multiple add-events from the directory as well, if multiple files are added, right? :/ – Anders Forsgren Aug 04 '11 at 15:16
  • I'm not quite sure what you're asking here. Do you want to populate the list at the point the code starts and then wait until everything is moved? Or would you populate the list gradually? – Chris Surfleet Aug 04 '11 at 15:18
  • @Chris: well, say a user selects a number of files and moves them to another subdirectory. In that case the event will trigger for each file (which will trigger both a delete and a create event, and I'm using the create event for detecting files moved). So even though each of the files the user moves in one operation will trigger the event, I only want to handle the event once all the files have been moved. – Anders Aug 04 '11 at 15:25
  • @Anders: well, I'm not sure I understand your question fully, I'm quite new to the FileSystemWatcher, but yes I assume what you're saying is correct, that there should be multiple triggers in both, so I'm not sure how to resolve it by that... – Anders Aug 04 '11 at 15:26
  • Ah, OK. Looks like Jay's answer is the way to go. I've always found the filesystemwatcher to be a bit flaky anyway! – Chris Surfleet Aug 04 '11 at 15:33
  • @Chris I agree! I've always thought the FSW should be easier to work with then it is. I found [this article](http://www.codeproject.com/KB/files/FileSystemWatcherChaos1.aspx) (and its second part) at CodeProject useful. – Jay Riggs Aug 04 '11 at 15:45

4 Answers4

11

Like Jay says: a timer is probably the only way to "group" events. The lock may be overkill but I don't like the idea of mutating a collection in a multithreaded situation (I think the events from the fswatcher are called on threads from a pool).

  public class Monitor : IDisposable
  {
     private List<string> filePaths;
     private ReaderWriterLockSlim rwlock;
     private Timer processTimer;
     private string watchedPath;
     private FileSystemWatcher watcher;

     public Monitor(string watchedPath)
     {
        filePaths = new List<string>();

        rwlock = new ReaderWriterLockSlim();

        this.watchedPath = watchedPath;
        InitFileSystemWatcher();
     }

     private void InitFileSystemWatcher()
     {
        watcher = new FileSystemWatcher();
        watcher.Filter = "*.*";
        watcher.Created += Watcher_FileCreated;
        watcher.Error += Watcher_Error;
        watcher.Path = watchedPath;
        watcher.IncludeSubdirectories = true;
        watcher.EnableRaisingEvents = true;
     }

     private void Watcher_Error(object sender, ErrorEventArgs e)
     {
        // Watcher crashed. Re-init.
        InitFileSystemWatcher();
     }

     private void Watcher_FileCreated(object sender, FileSystemEventArgs e)
     {
        try
        {
           rwlock.EnterWriteLock();
           filePaths.Add(e.FullPath);

           if (processTimer == null)
           {
              // First file, start timer.
              processTimer = new Timer(2000);
              processTimer.Elapsed += ProcessQueue;
              processTimer.Start();
           }
           else
           {
              // Subsequent file, reset timer.
              processTimer.Stop();
              processTimer.Start();
           }

        }
        finally
        {
           rwlock.ExitWriteLock();
        }
     }

     private void ProcessQueue(object sender, ElapsedEventArgs args)
     {
        try
        {
           Console.WriteLine("Processing queue, " + filePaths.Count + " files created:");
           rwlock.EnterReadLock();
           foreach (string filePath in filePaths)
           {
              Console.WriteLine(filePath);
           }
           filePaths.Clear();
        }
        finally
        {
           if (processTimer != null)
           {
              processTimer.Stop();
              processTimer.Dispose();
              processTimer = null;
           }
           rwlock.ExitReadLock();
        }
     }

     protected virtual void Dispose(bool disposing)
     {
        if (disposing)
        {
           if (rwlock != null)
           {
              rwlock.Dispose();
              rwlock = null;
           }
           if (watcher != null)
           {
              watcher.EnableRaisingEvents = false;
              watcher.Dispose();
              watcher = null;
           }
        }
     }

     public void Dispose()
     {
        Dispose(true);
        GC.SuppressFinalize(this);
     }

  }     

Remember to set the buffer size on your fswatcher AND implement "resurrection" of the fswatcher if it gets an error (i.e. bind the error event to a method that recreates the watcher).

Edit: note, the timer in this example is a System.Timers.Timer, not a System.Threading.Timer

Edit: Now contains error handling for the watcher, dispose logic.

Anders Forsgren
  • 10,827
  • 4
  • 40
  • 77
  • Ok, thanks, this does seem to be almost working, but I'm not sure what is going on with all the read locks and that stuff. It does get to the ProcessQueue the first time with all the files listed in the filepaths list. But then it starts going into some weird loop jumping back and forth in the code. What's wrong? Again, it does seem to have the right info the first time it gets there, but then it starts going back and forth... Don't understand why... – Anders Aug 04 '11 at 16:27
  • I updated the code sample (missed a start & stop), and tested it. Seems to work ok now. The lock is just there so there are no files added to the queue, during the processing of the queue. The fswatcher events that occur during the processing of the last batch simply wait and are added to the next batch. – Anders Forsgren Aug 04 '11 at 18:11
  • Thanks, it seems to work fine now! Just a couple of questions, if you don't mind: First, how do I handle recreating the watcher, in a try statement? Where? And what is going on here: processTimer.Elapsed += (o, ee) => ProcessQueue(); Thanks! – Anders Aug 04 '11 at 19:23
  • I changed the confusing (o,ee) to a normal (equivalent) delegate. I added the code to reinit the watcher after a failure, and some code to properly dispose the watcher. – Anders Forsgren Aug 04 '11 at 20:32
  • Ok, great. Just one more thing: when I try to raise an event after the list is finished so I can return it to the winform, it seems the ProcessQueue is called several times (one for each file?), whereas if I don't have an event triggered in there (just the foreach of your example) then it doesn't seem to be called more than once... Here's what I substituded the foreach with: var fileListEventArgs = new FileListEventArgs {FileList = filePaths}; OnFileListCreated(fileListEventArgs); Any idea why? – Anders Aug 04 '11 at 20:54
  • Correction, it wasn't the number of files, it seems to be called several more times than that... So the custom event gets triggered many times instead of just once... – Anders Aug 04 '11 at 20:57
  • No idea, I can't reproduce that. I get one invocation of the event handler if I add an event like you describe. Double check so you havent subscribed to the event more than once. If you subscribe to FileListCreated twice you will get the event handler called twice each time. – Anders Forsgren Aug 04 '11 at 21:16
  • No, it actually seems to go into an infinite loop, I have to stop the debugger to stop the calls... But I did get it to work if I move the clearing of the list and the event trigger into the finally statement. Is that wrong for some reason? It does seem to work fine anyway! See my update. – Anders Aug 04 '11 at 21:22
  • Decided this is answered now. I have a followup question if you think you can help with that too: http://stackoverflow.com/questions/6950071/filesystemwatcher-to-monitor-moved-files – Anders Aug 05 '11 at 09:24
5

I had to do the exact same thing. Use a System.Timers.Timer in your Monitor class and code its Elapsed event to process your List of files and clear the List. When the first item is added to your file List via the FSW events, start the Timer. When subsequent items are added to the list 'reset' the Timer by stopping and restarting it.

Something like this:

class Monitor
{
    FileSystemWatcher _fsw;
    Timer _notificationTimer;
    List<string> _filePaths = new List<string>();

    public Monitor() {
        _notificationTimer = new Timer();
        _notificationTimer.Elapsed += notificationTimer_Elapsed;
        // CooldownSeconds is the number of seconds the Timer is 'extended' each time a file is added.
        // I found it convenient to put this value in an app config file.
        _notificationTimer.Interval = CooldownSeconds * 1000;

        _fsw = new FileSystemWatcher();
        // Set up the particulars of your FileSystemWatcher.
        _fsw.Created += fsw_Created;
    }

    private void notificationTimer_Elapsed(object sender, ElapsedEventArgs e) {
        //
        // Do what you want to do with your List of files.
        //

        // Stop the timer and wait for the next batch of files.            
        _notificationTimer.Stop();
        // Clear your file List.
        _filePaths = new List<string>();
    }


    // Fires when a file is created.
    private void fsw_Created(object sender, FileSystemEventArgs e) {
        // Add to our List of files.
        _filePaths.Add(e.Name);

        // 'Reset' timer.
        _notificationTimer.Stop();
        _notificationTimer.Start();
    }
}
Jay Riggs
  • 53,046
  • 9
  • 139
  • 151
  • Thanks Jay, but I'm not sure. I may misunderstand you, but perhaps I didn't explain the situation enough. I don't see how this code will ever get called. Because what happens is, the application will be started only once, and then it will monitor all that happens in a directory. So the only thing that can make anything happen is if the Created event is triggered. I can't start the application and manually call the method that starts the timer. That would mean the code would only work for the first few seconds after starting the application? Or do I misunderstand you? – Anders Aug 04 '11 at 15:48
  • @Anders The Timer is stopped by default and starts when the FSW Created event fires. When the Timer Elapsed event fires the Timer stops itself. I'll add that my code is continuously running in a Windows Service. When the service starts the application starts and keeps running until the service stops. – Jay Riggs Aug 04 '11 at 15:58
  • Ok, I must be doing something wrong, because I tried it, slightly modified to work with the rest of my code, but it doesn't work. If I move one file the code is triggered. But only once. The next time I move a file the code doesn't trigger. And when I try moving several, it never reaches the elapsed event. See update! – Anders Aug 04 '11 at 16:13
3

Rx - throttle - makes this job easy. Here is a testable, reusable solution.

public class Files
{
     public static FileSystemWatcher WatchForChanges(string path, string filter, Action triggeredAction)
            {
                var monitor = new FileSystemWatcher(path, filter);

                //monitor.NotifyFilter = NotifyFilters.FileName;
                monitor.Changed += (o, e) => triggeredAction.Invoke();
                monitor.Created += (o, e) => triggeredAction.Invoke();
                monitor.Renamed += (o, e) => triggeredAction.Invoke();
                monitor.EnableRaisingEvents = true;

                return monitor;
            }
}

Lets merge events into a single sequence

  public IObservable<Unit> OnUpdate(string path, string pattern)
        {
            return Observable.Create<Unit>(o =>
            {
                var watcher = Files.WatchForChanges(path, pattern, () => o.OnNext(new Unit()));

                return watcher;
            });
        }

then finally, the usage

OnUpdate(path, "*.*").Throttle(Timespan.FromSeconds(10)).Subscribe(this, _ => DoWork())

u can obviously play with the throttle speed to suit your needs.

Weq
  • 81
  • 2
2

Also, set buffer size to larger than default to avoid buffer overflow. It happens when more than 25 files are dropped in source directory (in my test). If 200 files dropped, event handler is only called for few files, not all.

_watcher.InternalBufferSize = 65536; //Max size of buffer

HirenP
  • 61
  • 1
  • 6