Keep track of Parallel Foreach threads

Question

I am currently writing a simple WPF file copy app that copies files in parallel. So far it works great! It does everything I want it to do. The meat of the operation is in the following block of code:

Parallel.ForEach(Directory.GetFiles(dir).ToList(), file =>
{
    _destDetail.CurrOp = string.Format("Copying file: {0}", Path.GetFileName(file));
    File.Copy(file, file.Replace(_destDetail.Source, _destDetail.Dest), true);
    if (_destDetail.Progress < _destDetail.MaxProgress)
        _destDetail.Progress++;
});

I can implement ParallelOptions and limit the maximum number of threads to 4 as well, but I was wondering if there is a way to accurately keep track of what each thread would be doing in that case?

For example, say I have some portion of my UI that is dedicated to the current "Status" of the copy operation. I would want to have 4 rows in a Grid that each had a particular thread and which file it was currently copying.

I know I can use Interlocked to manipulate variables that are outside of the Parallel loop, but how would I keep track of thread-specific variables from inside of the Parallel loop and use those variables to keep the UI up to date on which thread is working on which file?

Just a FYI, `File.Copy` is a bad idea to do inside a `Parallel.ForEach`, you should only use `Parallel.ForEach` on CPU bound work, for IO bound work its scheduling algorithm will try to start too many tasks and it will most likely take longer to complete than if you just had copied the files in a normal `foreach` loop using a single thread. — Scott Chamberlain, Sep 22 '15 at 13:52

score 3 · Accepted Answer · edited May 23 '17 at 11:46

Instead of tracking the threads directly have the UI bind to a ObserveableCollection<ProgressDetail> each representing the progress, then in your loop have it add an item to the collection when it starts then remove it from the collection when it ends.

One thing you must be careful of is thread safety, ObseveableCollection is not thread safe so you must interact with it only in thread safe ways, the easiest way to do this is make all of the adds and removals of ProgressDetail objects on the UI thread. This also has the added benefit of capturing the SynchronizationContext of the UI thread when you create the Progress object.

public ObserveableCollection<ProgressDetail> ProgressCollection {get; private set;}

public void CopyFiles(string dir)
{

    var dispatcher = Application.Current.Dispatcher;
    Parallel.ForEach(Directory.GetFiles(dir).ToList(), file =>
    {
        ProgressDetail progressDetail = null;
        dispatcher.Invoke(() => 
        {
           // We make the `Progress` object on the UI thread so it can capture the  
           // SynchronizationContext during its construction.
           progressDetail = new ProgressDetail(file);
           ProgressCollection.Add(progressDetail);
        }

        XCopy.Copy(file, file.Replace(_destDetail.Source, _destDetail.Dest), 
                   true, false, progressDetail.ProgressReporter);

        dispatcher.Invoke(() => ProgressCollection.Remove(progressDetail);
    });

}

public sealed class ProgressDetail : INotifyPropertyChanged
{
    private double _progressPercentage;

    public ProgressDetail(string fileName)
    {
        FileName = fileName;
        ProgressReporter = new Progress<double>(OnProgressReported);
    }

    public string FileName { get; private set; }
    public IProgress<double> ProgressReporter { get; private set; }
    public double ProgressPercentage
    {
        get { return _progressPercentage; }
        private set
        {
            if (value.Equals(_progressPercentage)) return;
            _progressPercentage = value;
            OnPropertyChanged();
        }
    }

    private void OnProgressReported(double progress)
    {
        ProgressPercentage = progress;
    }

    public event PropertyChangedEventHandler PropertyChanged;
    private void OnPropertyChanged([CallerMemberName] string propertyName = null)
    {
        var temp = PropertyChanged;
        if(temp != null)
            temp(this, new PropertyChangedEventArgs(propertyName));
    }
}

See this answer for a example XCopy class that would copy with progress. I have made the assumption that the signature of Copy has been changed to

public static void Copy(string source, string destination, bool overwrite, bool nobuffering, IProgress<double> handler)

but I leave that actual change as a exercise for the reader.

UPDATE: I have updated the above code example to expose a public property ProgressPercentage which can be bound to and raises proper events. I have also moved the listening of the Progress event in to the internals of the ProgressDetail class.

You mention thread safety, then you mention binding. You can't really make binding thread safe, regardless of you locking the collection when adding/removing from it. — Blindy, Sep 21 '15 at 18:59
@Blindy I did not use any locking, I do all modifications that would cause a binding to fire on the UI thread, which is what my code does by using `dispatcher.Invoke` for the add and remove and [`Progress`](https://msdn.microsoft.com/en-us/library/hh193692(v=vs.110).aspx) for the reporting of the progress (`Progress` will raise it's [`ProgressChanged`](https://msdn.microsoft.com/en-us/library/hh137516(v=vs.110).aspx) using the `SynchronizationContext` at the time of its creation, if that was the UI thread then it raises the events on the UI thread). — Scott Chamberlain, Sep 21 '15 at 19:03

score 1 · Answer 2 · answered Sep 21 '15 at 18:03

The point about the Parallel library is that you don't know about the threads - which is quite applicable to this example. Your loop does some file IO and then some computation. While one of the threads is doing the IO, the thread is not in use and can be re-used to do the computation associated with one of the other files. This is also why it's better to leave the number of threads or concurrent tasks to the run-time: it knows better than you how many it can use.

Also as written _destDetail.Progress++; should really use Interlocked.Increment! (And calling .CurrOp is also open to race conditions.)

Keep track of Parallel Foreach threads

2 Answers2