5

I need my program to count the number of files on a disk drive.

What is the fastest way to do so?

Directory.GetFiles() is not an alternative, as it is very slow.

animaonline
  • 3,715
  • 5
  • 30
  • 57
  • What kind of an answer is that? There must be some way, we have NTFS journal, we have the WMI and so on lol – animaonline May 19 '12 at 17:34
  • 3
    It is a _comment_ on the question, not an answer. And you simply say the `GetFiles` "is slow". What is fast, in your definition? The question lacks detail. – Oded May 19 '12 at 17:35
  • if you have NTFS journal, then is possible to do Directory.GetFiles() on start and then using journal maintain counter. – volody May 19 '12 at 17:44
  • Did you try accessing the MFT directly? – HABO May 19 '12 at 17:54
  • @user92546 No, because my program needs to support FAT file systems too. :( – animaonline May 19 '12 at 17:58
  • @user92546 Windows doesn't allow direct access to the sectors – Cole Tobin May 19 '12 at 18:14
  • @ColeJohnson - The OP is disappointed with the performance of the OS, so an end run is an option. It ought to be possible to mount the volume privately and access it as an array of blocks. (There was no requirement stated to play nicely with others.) For those with strong stomachs some PInvoke code may be found [here](http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/c1550294-d121-4511-ac32-31551497f64e/). – HABO May 19 '12 at 19:52
  • working with MFT is fast, yes , but what I need is a solution that works both for FAT and NTFS... – animaonline May 19 '12 at 20:01
  • there is no solution which is faster than GetFiles, which also supports FAT file systems. the most time consuming factor is the random seek of the hard disc to read a new sector (good random read operations per seconds on magnetic hard drives are about 100 - 500), e.g. having 10.000 folders can make this quite slow – user287107 May 20 '12 at 12:45

3 Answers3

2

Did you try import kernel32.dll and use it?

There is a good implementation example someone posted before here : https://stackoverflow.com/a/724184/912851. It might be worthwhile looking at.

Edit: The fastest one I saw in my life is this application. It uses ntfs journals. and within seconds it lists millions of files on my harddisk. I think they have a sdk and sources on C++ or c. Maybe you can create a managed dll and use on c#?

Community
  • 1
  • 1
Nesim Razon
  • 9,684
  • 3
  • 36
  • 48
  • 1
    I was curious so I made a small code to compare both methods on directory tree with 42k files: GetFiles=1.2seconds, this method=650ms. So it's already 2x faster... As Oded already write above, IO is slow, you'll not have immediate result for hundred of thousands files... – Fabske May 19 '12 at 18:00
0

you could read out the drive USN journal, this is very fast but you need administrator rights

user287107
  • 9,286
  • 1
  • 31
  • 47
-2
Func<string, int> files = null;
files = p => Directory.GetFiles(p).Length() + Directory.GetDirectories(p).Select(p1 => files(p1));

int count = files(@"c:\");

Horrible but linqy!

Matthew Abbott
  • 60,571
  • 9
  • 104
  • 129