Getting files recursively: skip files/directories that cannot be read?

Question

I want to get all of the files in a directory in an array (including the files in subfolders)

string[] filePaths = Directory.GetFiles(@"c:\",SearchOption.AllDirectories);

The problem with this is: If an exception is thrown the entire command stops. Is there a better way to do this so that if a folder cannot be accessed it will just skip over it?

Judging by the `@"c:\"`, I'm thinking he's asking if the method call can finish its work and get all the directories where access is granted. — hmqcnoesy, Aug 13 '12 at 01:03
The non-overloaded [`GetFiles` method shows an example of recursing manually](http://msdn.microsoft.com/en-us/library/07wt70x2.aspx) and it wouldn't be hard to add exception handling to it .. I am *assuming* the issue comes up with not being able to *read* sub-directories; and that a `Directory.GetFiles(.., without_recursive_search)` will operate as all-or-nothing for the *specific directory* as expected. — , Aug 13 '12 at 01:09
(Although, that example is simplistic: The result is side-effect only and "Bad Things" will happen if there exist recursive directory junctions or links ..) — , Aug 13 '12 at 01:12

score 7 · Accepted Answer · answered Aug 13 '12 at 01:12

7

You'd probably have to do a bit more typing yourself then, and write a directory walker like this one:

    public static string[] FindAllFiles(string rootDir) {
        var pathsToSearch = new Queue<string>();
        var foundFiles = new List<string>();

        pathsToSearch.Enqueue(rootDir);

        while (pathsToSearch.Count > 0) {
            var dir = pathsToSearch.Dequeue();

            try {
                var files = Directory.GetFiles(dir);
                foreach (var file in Directory.GetFiles(dir)) {
                    foundFiles.Add(file);
                }

                foreach (var subDir in Directory.GetDirectories(dir)) {
                    pathsToSearch.Enqueue(subDir);
                }

            } catch (Exception /* TODO: catch correct exception */) {
                // Swallow.  Gulp!
            }
        }

        return foundFiles.ToArray();
    }

answered Aug 13 '12 at 01:12

ikh

2,336
2
19
28

1

Things of note: 1) This is a BFS implemenetation, not a DFS. 2) This won't deal with recursive structures (junction point or hard/soft links in NTFS) 3) `ToArray` seems silly. – Aug 13 '12 at 02:30
1

@pst Agree with points 2 and 3 (`ToArray` was to produce the same `string[]` as in the question and is not at all necessary), but does point 1 make much difference if all accessible subdirectories need to be traversed? – ikh Aug 13 '12 at 02:45
1

@ikh I'm not saying it's bad at all, just that it is :) It is the difference between traversing in order of: `C:\a\ C:\b\ C:\a\aa\ ` (breadth-first) and `C:\a\ C:\a\aa\ C:\b\ ` (depth-first) -- Some might be expecting the other (DFS) behavior so it's something to keep in mind. – Aug 13 '12 at 03:38
This worked perfectly!This is much slower than the built-in version - but of course they don't handle unauth files. – Michael K Dec 14 '18 at 15:06

jw_ · Answer 2 · 2020-02-06T10:00:49.613

Directory.GetFiles can't skip directory symbol links which often cause loops and then exceptions.

So based on @iks's answer and Check if a file is real or a symbolic link, here is a version that deliver the result on the go like Directory.EnumerateFiles does:

    public static IEnumerable<string> FindAllFiles(string rootDir)
    {
        var pathsToSearch = new Queue<string>();


        pathsToSearch.Enqueue(rootDir);

        while (pathsToSearch.Count > 0)
        {
            var dir = pathsToSearch.Dequeue();
            var foundFiles = new List<string>();
            try
            {
                foreach (var file in Directory.GetFiles(dir))
                    foundFiles.Add(file);

                foreach (var subDir in Directory.GetDirectories(dir))
                {
                    //comment this if want to follow symbolic link
                    //or follow them conditionally
                    if (IsSymbolic(subDir)) continue;
                    pathsToSearch.Enqueue(subDir);
                }
            }
            catch (Exception) {//deal with exceptions here
            }
            foreach (var file in foundFiles) yield return file;
        } 


    }

    static private bool IsSymbolic(string path)
    {
        FileInfo pathInfo = new FileInfo(path);
        return pathInfo.Attributes.HasFlag(System.IO.FileAttributes.ReparsePoint);
    }

    static public void test()
    {
        string root = @"D:\root";
        foreach (var fn in FindAllFiles(root)
            .Where(x=>
            true    //filter condition here
            ))
        {
            Debug.WriteLine(fn);
        }
    }

A ReparsePoint is not not necessarily a Symbolic link! This will get "false positives". To recognize real symbolic links you have to use native APIs. A full working implementation can be found here: https://www.codeproject.com/Articles/15633/Manipulating-NTFS-Junction-Points-in-NET — marsh-wiggle, Oct 19 '20 at 14:03

score -1 · Answer 3 · answered Aug 13 '12 at 01:18

-1

try this :

DirectoryInfo directory = new DirectoryInfo(@"c:\");
        DirectoryInfo[] folders = directory.GetDirectories("*", SearchOption.AllDirectories);

        List<string> files = new List<string>();
        foreach (DirectoryInfo info in folders)
        {
            foreach (FileInfo file in info.GetFiles())
            {
                files.Add(file.Name);
            }
        }

answered Aug 13 '12 at 01:18

Habib Zare

1,206
8
17

1

What happens if there is a folder `C:\NoPermissionsToRead`? – Aug 13 '12 at 03:43

Habib Zare · Answer 4 · 2012-08-13T01:38:59.397

-1

or try this one :

        DirectoryInfo dirs = new DirectoryInfo(@"c:\");
        List<string> filenames = (from i in dirs.GetFiles("*", SearchOption.AllDirectories)
                                  select i.Name).ToList();

or file names without extension :

        DirectoryInfo dirs = new DirectoryInfo@"c:\");
        List<string> filenames = (from i in dirs.GetFiles("*", SearchOption.AllDirectories)
                                  select System.IO.Path.GetFileNameWithoutExtension(i.Name)).ToList();

edited Aug 13 '12 at 01:38

answered Aug 13 '12 at 01:27

Habib Zare

1,206
8
17

1

How will this handle/act to cases where the directory cannot be read? – Aug 13 '12 at 03:43

Getting files recursively: skip files/directories that cannot be read?

4 Answers4

Linked

Related