0

I have code with Parallel.Foreach which is processing files and doing some operation on each file in parallel. 

Parallel.ForEach(lstFiles, file=>
{
 // Doing some operation on file
 // Skip file and move to next if it is taking too long
});

I want to skip a file and move to next file (but don't want to exit the Parallel.Foreach) if a particular file is taking too long (say 2 mins). Is there any way in Parallel.Foreach to check the time taken by thread to process a single file.

Thanks

Gurmeet
  • 3,094
  • 4
  • 19
  • 43
  • You could await a task inside the foreach block and wait for the task with WhenAny like is shown in https://stackoverflow.com/questions/4238345/asynchronously-wait-for-taskt-to-complete-with-timeout. – blit Nov 24 '17 at 04:55
  • 2
    `I want to skip a file and move to next file` - This statment is confusing. `Parallel.ForEach` is processing all files in the list in paralell, not 1 at a time. If you decide to just stop processing a particular file, that's all you need to do. The others will keep going concurrently. – Ashley Pillay Nov 24 '17 at 05:41

1 Answers1

1

I'd suggest you don't use Parallel.ForEach and instead use Mirosoft's extremely more powerful Reactive Framework. Then you can do this:

var query =
    from file in lstFiles.ToObservable()
    from result in Observable.Amb(
        Observable.Start(() => SomeOperation(file)).Select(_ => true),
        Observable.Timer(TimeSpan.FromMinutes(2.0)).Select(_ => false))
    select new { file, result };

IDisposable subscription =
    query
        .Subscribe(x =>
        {
            /* do something with each `new { file, result }`
               as they arrive. */
        }, ex =>
        {
            /* do something if an error is encountered */
            /* (stops processing on first error) */
        }, () =>
        {
            /* do something if they have all finished successfully */
        })

This is all done in parallel. The Observable.Amb operator starts the two observables defined in its argument list and takes the value from which ever of the two produces a value first - if it's the Start observable it has processed your file and if it's the Timer observable then 2.0 minutes has elapsed without a result from the file.

If you want to stop the processing when it is half-way through then just call subscription.Dispose().

Use NuGet "System.Reactive" to get the bits.

The query in lambda form as per request in comments:

var query =
    lstFiles
        .ToObservable()
        .SelectMany(
            file =>
                Observable.Amb(
                    Observable.Start(() => SomeOperation(file)).Select(_ => true),
                    Observable.Timer(TimeSpan.FromMinutes(2.0)).Select(_ => false)),
            (file, result) => new { file, result });
Enigmativity
  • 113,464
  • 11
  • 89
  • 172