Parallel LINQ UnauthorizedAccessException when Aggregating results

Question

I am searching in parallel using LINQ to find pattern matching files.

public class ParallelLinq
{
    public IList<string> SearchFolders = new List<string>
    {
        @"C:\Windows" //can be multiple
    };

    protected virtual IEnumerable<string> GetFiles(string path, string[] searchPatterns, SearchOption searchOption = SearchOption.AllDirectories)
    {
        return searchPatterns.AsParallel()
            .SelectMany(searchPattern =>
            {
                try
                {
                    return Directory.EnumerateFiles(path, searchPattern, searchOption);
                }
                catch (Exception ex) //catch UnauthoizedException/IOExceptions
                {
                    return Enumerable.Empty<string>();
                }
            });
    }

    public IEnumerable<string> Find(IList<string> patterns)
    {
        var testResultFiles = Enumerable.Empty<string>();

        if (!SearchFolders.Any() || !patterns.Any())
        {
            return testResultFiles;
        }

        testResultFiles = SearchFolders.AsParallel().Aggregate(testResultFiles, (current, folder) => current.Union(GetFiles(folder, patterns.ToArray())));

        return testResultFiles;
    }
}

However when I try evaluate the values I am running into System.UnauthorizedAccessException: Access to the path 'C:\Windows\appcompat\Programs' is denied.

var plinq = new ParallelLinq();
var res = plinq.Find(new List<string> { "*.dll" });
Console.WriteLine("Linq Count: " + res.Count());

While these exceptions are expected, How can we catch them and continue ahead?

Full Exception:

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.UnauthorizedAccessException: Access to the path 'C:\Windows\appcompat\Programs' is denied. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileSystemEnumerableIterator1.AddSearchableDirsToStack(SearchData localSearchData) at System.IO.FileSystemEnumerableIterator1.MoveNext() at System.Linq.Parallel.SelectManyQueryOperator3.SelectManyQueryOperatorEnumerator1.MoveNext(TOutput& currentElement, Pair2& currentKey) at System.Linq.Parallel.PipelineSpoolingTask2.SpoolingWork() at System.Linq.Parallel.SpoolingTaskBase.Work() at System.Linq.Parallel.QueryTask.BaseWork(Object unused) at System.Linq.Parallel.QueryTask.<>c.<.cctor>b__10_0(Object o) at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of inner exception stack trace --- at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose) at System.Linq.Parallel.AsynchronousChannelMergeEnumerator1.MoveNextSlowPath() at System.Linq.Parallel.AsynchronousChannelMergeEnumerator1.MoveNext()
at System.Linq.Parallel.QueryOpeningEnumerator1.MoveNext() at System.Linq.Enumerable.<UnionIterator>d__671.MoveNext() at System.Linq.Enumerable.Count[TSource](IEnumerable`1 source)

so what you have is not catching the error? Its unclear whether you are asking if this can be optimised, or its not working at all as soon as it hits an access denied error — TheGeneral, Mar 11 '19 at 07:00
@MichaelRandall it's not working. It's throwing the `Exception` and returning zero results — RaceBase, Mar 11 '19 at 08:04
Directory.EnumerateFiles returns IEnumerable, so "delayed" sequence, and an exception is being thrown later when sequence is being materialized (at enumerator's MoveNext). Please use this question for traversing through subfolders and avoiding inaccessible subfolders: https://stackoverflow.com/questions/172544/ignore-folders-files-when-directory-getfiles-is-denied-access — Renat, Mar 11 '19 at 08:50

score 0 · Answer 1 · answered Mar 29 '19 at 17:56

public class ParallelLinq
{
    public IList<string> SearchFolders = new List<string>
    {
        @"C:\Windows" //can be multiple
    };

    private static string[] TryGetTopDirectories(string path)
    {
        try
        {
            return Directory.GetDirectories(path, "*", SearchOption.TopDirectoryOnly);
        }
        catch
        {
            return new string[0];
        }
    }

    private static IEnumerable<string> GetSubfolders(string path, SearchOption searchOption)
    {
        if (searchOption == SearchOption.TopDirectoryOnly)
        {
            return TryGetTopDirectories(path);
        }
        else
        {
            var topFolders = TryGetTopDirectories(path);
            return topFolders.Concat(
                topFolders.SelectMany(subFolder => GetSubfolders(subFolder, searchOption)));
        }
    }

    protected virtual ParallelQuery<string> GetFiles(string path, string[] searchPatterns, SearchOption searchOption = SearchOption.AllDirectories)
    {
        return GetSubfolders(path, searchOption).AsParallel()
            .SelectMany(subfolder =>
            {
                try
                {
                    return searchPatterns.SelectMany(searchPattern => Directory.EnumerateFiles(subfolder, searchPattern)).ToArray();
                }
                catch (Exception ex) //catch UnauthoizedException/IOExceptions
                {
                    return Enumerable.Empty<string>();
                }
            });
    }

    public IEnumerable<string> Find(IList<string> patterns)
    {
        var testResultFiles = Enumerable.Empty<string>();

        if (!SearchFolders.Any() || !patterns.Any())
        {
            return testResultFiles;
        }

        testResultFiles = SearchFolders.AsParallel().Aggregate(testResultFiles, (current, folder) => current.Union(GetFiles(folder, patterns.ToArray())));

        return testResultFiles;
    }
}

score -1 · Answer 2 · answered Mar 11 '19 at 08:39

-1

Seems like the path 'C:\Windows\appcompat\Programs' is preventing the program from creating files. It can be resolved by adding extra permission in the folder itself.

How to add permission to folder manually

answered Mar 11 '19 at 08:39

John Carlo Diocadiz

29
4

Parallel LINQ UnauthorizedAccessException when Aggregating results

2 Answers2