0

i need to watch a folder for new files where at max 10k files are created within 3hrs. These is the first time am using FileSystemWatcher so i'm bother whether it can handle so many files at a time. So using filesystemWatcher is best approach. Or polling the folder continuously is best approach?

So please suggest me the best approach, even other than these two.

Thanks in Advance.

sandeep
  • 2,862
  • 9
  • 44
  • 54
  • Be aware that `FileSystemWatcher` isn't 100% bulletproof. http://stackoverflow.com/questions/239988/filesystemwatcher-vs-polling-to-watch-for-changes – xanatos Oct 29 '11 at 12:03

1 Answers1

4

10000 files in 3 hours is only about 1 per second (although obviously it might be burstier). I doubt that that would cause a problem for FileSystemWatcher. Have you tried it?

I would be slightly concerned about how well a directory with 10,000 files in would perform anyway though - it probably depends on the file system. Are you able to move "processed" files to a different directory as you go?

Polling is certainly an option - although with both approaches, you'll need to be careful about how you handle new files, in case they're still being written. (One option is to write to a different directory and then atomically rename the file - or to use filename extensions for the same sort of effect, e.g. writing to foo.tmp and then renaming to foo.txt, and only processing .txt files.)

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • I have not yet tried. Is there any alternative solution for this, is polling folder a good approach? – sandeep Oct 29 '11 at 12:01
  • @sandeep: Polling could end up being a pain if you've actually got 10,000 files in later on - it *may* be relatively expensive to list the directory at that point. It's an option though. I think the handling of partially-written files is likely to be something which causes you more pain unless you're careful though. – Jon Skeet Oct 29 '11 at 12:02
  • Actually at max within 3 hrs i can get 10,000 files, but rest of the hrs of the day i can get only 1k files. – sandeep Oct 29 '11 at 12:06
  • @sandeep: Right, so you build the solution so that it will cope with the maximum. – Jon Skeet Oct 29 '11 at 13:29