DirectoryInfo.EnumerateFiles(...) causes UnauthorizedAccessException (and other exceptions)

Question

I have recently had a need to Enumerate an entire file system looking for specific types of files for auditing purposes. This has caused me to run into several exceptions due to having limited permissions on the file system to be scanned. Among them, the most prevalent have been UnauthorizedAccessException and much to my chagrin, PathTooLongException.

These would not normally be an issue except that they invalidate the IEnumerable, preventing me from being able to complete the scan.

Answer 1

In order to solve this problem, I have created a replacement File System Enumerator. Although it may not be perfect, it performs fairly quickly and traps the two exceptions that I have run into. It will find any directories or files that match the search pattern passed to it.

// This code is public domain
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using log4net;

public class FileSystemEnumerable : IEnumerable<FileSystemInfo>
{
    private ILog _logger = LogManager.GetLogger(typeof(FileSystemEnumerable));

    private readonly DirectoryInfo _root;
    private readonly IList<string> _patterns;
    private readonly SearchOption _option;

    public FileSystemEnumerable(DirectoryInfo root, string pattern, SearchOption option)
    {
        _root = root;
        _patterns = new List<string> { pattern };
        _option = option;
    }

    public FileSystemEnumerable(DirectoryInfo root, IList<string> patterns, SearchOption option)
    {
        _root = root;
        _patterns = patterns;
        _option = option;
    }

    public IEnumerator<FileSystemInfo> GetEnumerator()
    {
        if (_root == null || !_root.Exists) yield break;

        IEnumerable<FileSystemInfo> matches = new List<FileSystemInfo>();
        try
        {
            _logger.DebugFormat("Attempting to enumerate '{0}'", _root.FullName);
            foreach (var pattern in _patterns)
            {
                _logger.DebugFormat("Using pattern '{0}'", pattern);
                matches = matches.Concat(_root.EnumerateDirectories(pattern, SearchOption.TopDirectoryOnly))
                                 .Concat(_root.EnumerateFiles(pattern, SearchOption.TopDirectoryOnly));
            }
        }
        catch (UnauthorizedAccessException)
        {
            _logger.WarnFormat("Unable to access '{0}'. Skipping...", _root.FullName);
            yield break;
        }
        catch (PathTooLongException ptle)
        {
            _logger.Warn(string.Format(@"Could not process path '{0}\{1}'.", _root.Parent.FullName, _root.Name), ptle);
            yield break;
        } catch (System.IO.IOException e)
        {
            // "The symbolic link cannot be followed because its type is disabled."
            // "The specified network name is no longer available."
            _logger.Warn(string.Format(@"Could not process path (check SymlinkEvaluation rules)'{0}\{1}'.", _root.Parent.FullName, _root.Name), e);
            yield break;
        }


        _logger.DebugFormat("Returning all objects that match the pattern(s) '{0}'", string.Join(",", _patterns));
        foreach (var file in matches)
        {
            yield return file;
        }

        if (_option == SearchOption.AllDirectories)
        {
            _logger.DebugFormat("Enumerating all child directories.");
            foreach (var dir in _root.EnumerateDirectories("*", SearchOption.TopDirectoryOnly))
            {
                _logger.DebugFormat("Enumerating '{0}'", dir.FullName);
                var fileSystemInfos = new FileSystemEnumerable(dir, _patterns, _option);
                foreach (var match in fileSystemInfos)
                {
                    yield return match;
                }
            }
        }
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

The usage is fairly simple.

//This code is public domain
var root = new DirectoryInfo(@"c:\wherever");
var searchPattern = @"*.txt";
var searchOption = SearchOption.AllDirectories;
var enumerable = new FileSystemEnumerable(root, searchPattern, searchOption);

People are free to use it if they find it useful.

Answer 2

Here's another way, manage your own enumeration iteration:

IEnumerator<string> errFiles=Directory.EnumerateFiles(baseDir, "_error.txt", SearchOption.AllDirectories).GetEnumerator();
while (true)
{
    try
    {
        if (!errFiles.MoveNext())
            break;
        string errFile = errFiles.Current;
        // processing
    } catch (Exception e)
    {
        log.Warn("Ignoring error finding in: " + baseDir, e);
    }
}