Stopping Parallel.ForEach if one of threads performs more than N minutes

Question

I'm looking for a solution for stopping Parallel.ForEach if one of threads performs more than 2 minutes.

The next solution I think is not very good because of x2 extra threads:

Parallel.ForEach(items, (item, opt) =>
{
    var thread = new Thread(() => { /* a process */ });
    thread.Start();

    bool finished = thread.Join(TimeSpan.FromMinutes(2));
    if (!finished)
    {
        thread.Abort();
        opt.Stop();
    }
});

Do you know a better solution for the issue?

I'd use an array of `Task` with a cancellation token and timeout. You may be asking too much of `Parallel.ForEach`. Its also very odd that you are using `Parallel.ForEach` to kick off threads that do work... — Ron Beyer, Aug 11 '15 at 15:25
@RonBeyer, can you post a solution here using an array of `Task`? — Neshta, Aug 11 '15 at 16:28
Its not that simple because the work needs to be batched and broken into the tasks. It depends on how many items are in your `items` collection. — Ron Beyer, Aug 11 '15 at 19:29
@RonBeyer, the collection has about 10^6 items. And time for calculation of one item can vary. Maybe you can suggest me a solution for tracking all active threads from the main thread? — Neshta, Aug 12 '15 at 07:49
Use `CancellationTokenSource` for the cancellation for the Tasks in this case, and set flag inside one task for being able to say which task is currently running — VMAtm, Aug 12 '15 at 08:47
`Thread.Abort` is dangerous and could leave your program in a corrupted state, is it possible to re-write your long running code using a `CancelationToken` and have it cooperatively cancel? — Scott Chamberlain, Aug 12 '15 at 13:18
@ScottChamberlain, No, it's impossible. Functionality of a process is too hard to modify. And I don't need to cancel other threads if one of them is performed too long. All items are independent. — Neshta, Aug 12 '15 at 13:29
You should really read through http://stackoverflow.com/questions/1559255/whats-wrong-with-using-thread-abort Even if all you do is abort a single thread you can still screw a lot of stuff up. For example a big problem is if your code uses `lock` the locks will not be released if the abort happened inside the lock!. The only "Safe" way to do a non cooperative abort is move the code to a separate exe then call `Process.Kill` on the exe, but this adds a lot of overhead. — Scott Chamberlain, Aug 12 '15 at 13:32
@ScottChamberlain, I understand you. I will try to refactor code for using a `CancelationToken`. Thank you for your advice! — Neshta, Aug 12 '15 at 13:47

score 1 · Answer 1 · edited May 23 '17 at 11:51

First of all, I want to note that Parallel class will not create a thread for each of your item, it will use default ThreadPool, which has by default number of threads equal to processor's cores count. Other problem in your code is that you do not stop all the tasks after 2 minutes of working, you only cancel the one which you've waited for two minutes.

I suggest you remove the Thread usage from your code, and create an array of Tasks with single CancellationToken for them with a Timeout for it or with a timeout for TaskFactory, and start them all. Also your code should explicitly check the token for cancellation pending.

So your code could be something like this:

var cts = new CancellationTokenSource();
// two minutes in milliseconds
cts.CancelAfter(2000 * 60);

var tasks = new List<Task>();
foreach (var item in items)
{
    // this is needed because of closures work in C#
    var localItem = item;
    tasks.Add(Task.Run(() => 
        { /* a process with a localItem here */ 
            // this check should be repeated from time to time in your calculations
            if (cts.Token.IsCancellationRequested)
            {
                cts.Token.ThrowIfCancellationRequested();
            }
        }
    // all tasks has only one token
    , cts.Token)

}
// this will cancel all tasks after 2 minutes from start
Task.WaitAll(tasks.ToArray(), TimeSpan.FromMinutes(2));
// this will cancel all tasks if one of them will last more than 2 minutes
Task.WaitAll(tasks.ToArray());

Update:

As you said that the each item is independent, you can create CancellationTokenSource for each task, but, as @ScottChamberlain noted, in this case too many tasks will run in the same time. You can write your own TaskScheduler, use some Semafor (or it's slim version) or simply use the Parallel class with ParallelOptions.MaxDegreeOfParallelism correctly set.

One problem with this, you will now spin up `items.Count` tasks all at once (the op noted there are 10^6 items in the collection). the original `Parallel.ForEach` method would rate limit itself. A littile reorganizing plus a `SemaphoreSlim` you could re-create `ParallelOptions.MaxDegreeOfParallelism`. — Scott Chamberlain, Aug 12 '15 at 13:15

Stopping Parallel.ForEach if one of threads performs more than N minutes

1 Answers1