Cancel background running jobs in Parallel.foreach

Question

I have a parallel foreach for few api calls. Api will return some data and I can process it. Lets say from front end, I call to ProcessInsu method and suddenly the user change his mind and go to other page. (There should be a clear button and user can exit the page after clicking clear button.) After calling ProcessInsu method, it will run APIs in background. It is waste of resources because, the user already change his mind and do other work. I need some methodology to cancel background running jobs.

public async Task ProcessInsu(InsuranceAccounts insuranceCompAccounts,string resourceId)
{
     ParallelOptions parallelOptions = new ParallelOptions();
     parallelOptions.MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling((Environment.ProcessorCount * 0.75) * 2.0));

     Parallel.ForEach(insuranceCompAccounts, parallelOptions, async (insuranceComp) =>
    {
        await Processor.Process(resourceId,insuranceComp); //When looping, From each call, it will call to different different insurance companies.              
    });

}

I tried this sample code and I could not do my work with that. Any expert can guide me to do this?

Don't use `Parallel.ForEach` with IO bound workloads, or the async and await pattern, it is not suitable, use `Task.WhenAll` or something else. If you want to cancel a task/workload, you can use a cancelation token cooperatively. — TheGeneral, Nov 19 '20 at 04:18
With certain very narrow (and probably inappropriate) exceptions, code cannot "cancel" other code that is running. Think of all the problems that could create! Instead, cancellation is cooperative-- code has to cancel *itself*, e.g. by checking a cancellation token or flag at regular intervals and exiting if the flag has been set. Once the cancel signal is detected, the proper way to exit is to [call `LoopState.Break`](https://stackoverflow.com/a/19305042/2791540). — John Wu, Nov 19 '20 at 04:24
You need to show why "you could not do my work with that", since cooperative cancellation with tokens is the way to go. Processor.Process should accept and handle the token (parallel.foreach loop is quite useless here as stated above). — Evk, Nov 19 '20 at 04:28
The `Parallel.ForEach` [is not async-friendly](https://stackoverflow.com/questions/15136542/parallel-foreach-with-asynchronous-lambda). — Theodor Zoulias, Nov 20 '20 at 04:14
Is it an option to pass a `CancellationToken` as an argument of the `Processor.Process` method, and honor a possible signal by immediately cancelling the processing? — Theodor Zoulias, Nov 20 '20 at 04:18
The property [`ParallelOptions.CancellationToken`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.paralleloptions.cancellationtoken) could be relevant if you used the `Parallel.ForEach` for its intended purpose, which is to parallelize heavy CPU-bound work. — Theodor Zoulias, Nov 20 '20 at 04:21

score 1 · Answer 1 · answered Nov 24 '20 at 20:57

As others have noted, you can't use async with Parallel - it won't work correctly. Instead, you want to do asynchronous concurrency as such:

public async Task ProcessInsu(InsuranceAccounts insuranceCompAccounts, string resourceId)
{
  var tasks = insuranceCompAccounts.Select(async (insuranceComp) =>
  {
    await Processor.Process(resourceId, insuranceComp);
  }).ToList();
  await Task.WhenAll(tasks);
}

Now that the code is corrected, you can add cancellation support. E.g.:

public async Task ProcessInsu(InsuranceAccounts insuranceCompAccounts, string resourceId)
{
  var cts = new CancellationTokenSource();

  var tasks = insuranceCompAccounts.Select(async (insuranceComp) =>
  {
    await Processor.Process(resourceId, insuranceComp, cts.Token);
  }).ToList();
  await Task.WhenAll(tasks);
}

When you are ready to cancel the operation, call cts.Cancel();.

Note that Process now takes a CancellationToken. It should pass this token on to whatever I/O APIs it's using.

Cancel background running jobs in Parallel.foreach

1 Answers1