File Search in C#

Question

I have made this code:

string[] paths = GetFiles(dir).ToArray();

int numberOfFiles = paths.Length;
int i = 0;
while (i < numberOfFiles - 1)
{
    scanfile(paths[i]);
    i++;
}

And for the GetFiles:

static IEnumerable<string> GetFiles(string path) {
Queue<string> queue = new Queue<string>();
queue.Enqueue(path);
while (queue.Count > 0)
{
    path = queue.Dequeue();
    try
    {
        foreach (string subDir in Directory.GetDirectories(path))
        {
            queue.Enqueue(subDir);
        }
    }
    catch (Exception ex)
    {
        Console.Error.WriteLine(ex);
    }
    string[] files = null;
    try
    {
        files = Directory.GetFiles(path);
    }
    catch (Exception ex)
    {
        Console.Error.WriteLine(ex);
    }
    if (files != null)
    {
        for (int i = 0; i < files.Length; i++)
        {
            yield return files[i];
        }
    }
}
}

public string scanfile(string path)
    {
        int offset = 0;
        int length = 0;
        byte[] buffer;
        var variable1 = new StringBuilder();

        FileInfo fi = new FileInfo(path);
        length = (int)fi.Length;


            using (var mmf1 = MemoryMappedFile.CreateFromFile(path,
                FileMode.OpenOrCreate, null, offset + length))
            {
                // Create reader to MMF
                using (var reader = mmf1.CreateViewAccessor(300,
                4000, MemoryMappedFileAccess.Read))
                {
                    // Read from MMF
                    buffer = new byte[4000];
                    reader.ReadArray<byte>(0, buffer, 0, 4000);
                }
            }

        return variable1.ToString();

    }

P.S.: Get Files I took it from Stack Overflow too.

But, it's very slow if i use it for many files in a directory because it calculates first the files in the directory.

Can you help me to optimize these codes?

That code won't just be "very slow" - it will be infinite, as you're never changing the value of `i` or `numerOfFiles`. That suggests this isn't your real code, which makes it hard to diagnose your problem much further. — Jon Skeet, Oct 01 '12 at 13:35
Also, you don't seem to show what `scanfile` or `GetFiles` do, and also why do you need it in an array? — Arran, Oct 01 '12 at 13:36
I would suggest this answer: http://stackoverflow.com/questions/724148/is-there-a-faster-way-to-scan-through-a-directory-recursively-in-net/724184#724184 — Karel Burda, Oct 01 '12 at 13:36
Which version of .NET are you using? Can you give the scanfile() method to a thread, possibly from the thread pool? — JeffFerguson, Oct 01 '12 at 13:36

score 2 · Answer 1 · answered Oct 01 '12 at 13:47

2

Okay, let's see if we can simplify this a bit. Let's use this code to get all of the files:

// Get list of files in the specific directory.
// ... Please change the first argument.
string[] files = Directory.GetFiles("{root path}",
    "*.*",
    SearchOption.AllDirectories);

Note that AllDirectories is documented as such:

Includes the current directory and all its subdirectories in a search operation. This option includes reparse points such as mounted drives and symbolic links in the search.

And now let's just use a simple foreach loop to get through them:

foreach (var file in files)
{
    // ... do something
}

Is there a reason this won't work?

answered Oct 01 '12 at 13:47

Mike Perrenoud

66,820
29
157
232

It doesn't works with an error : "Access to the path is denied" ?! – Michael Oct 01 '12 at 13:49
First, it is not recursive and second thing is, access denied error is a file system issue and not of your code. You cannot resolve it using code. Either you have to change the file system permission or run the code with proper user privileges so that it can access all files. – Murtuza Kabul Oct 01 '12 at 13:54
Okay. So, it is an error because of my computer? And if I run it on another computer, it will work? – Michael Oct 01 '12 at 13:56
@MurtuzaKabul, I'm not sure how you can indicate that it's __not__ recursive ... have you actually prototyped that? Are you indicating that the documentation from Microsoft is a flat-out lie? – Mike Perrenoud Oct 01 '12 at 14:20
@Michael, the issue surrounding the permissions error is because you don't have access to one of the directories it's trying to search. It's likely a sub directory because I'm assuming you have access to the root. – Mike Perrenoud Oct 01 '12 at 14:21

score 1 · Answer 2 · answered Oct 01 '12 at 13:37

1

You should iterate through the result of GetFiles using foreach loop instead of fetching it as an array and then scanning individual file.

answered Oct 01 '12 at 13:37

Murtuza Kabul

6,438
6
27
34

but if I use foreach loop, it has an error "Access path is denied" – Michael Oct 01 '12 at 13:39
I have used the foreach method plenty of times. It works well and is very fast! – Joe DF Oct 01 '12 at 13:42
You are calling the ToArray method which is not required. Further, the iterator has some optimizations which will make it slightly faster. The "Access path is denied" error has to do nothing with the foreach loop. Make sure that you have full permission to the files you are trying to access – Murtuza Kabul Oct 01 '12 at 13:42
@micheal show the code.. Pastebin? – Joe DF Oct 01 '12 at 13:43
how to get the full permission? – Michael Oct 01 '12 at 13:43
Post the new code so I can find the exact bug – Murtuza Kabul Oct 01 '12 at 13:44

score 1 · Answer 3 · answered Oct 01 '12 at 14:13

1

Simply take a look into Directory.EnumerateFiles. Their is also an example on how to iterate over all *.txt files and get all lines containing Microsoft.

The example also takes care for the unauthorized access exception by simply skipping this file.

answered Oct 01 '12 at 14:13

Oliver

43,366
8
94
151

File Search in C#

3 Answers3