1

I have code like this:

var listAction = new List<Task>();
...

foreach (var symbol in chunk)
{
    listAction.Add(Task.Run(() => DoBooster(symbol, ...)));
}

int failed = 0;
Task t = Task.WhenAll(listAction);
try
{
    t.Wait();
}
catch { }

if (t.Status == TaskStatus.RanToCompletion)
{
    Console.WriteLine("All Boosters succeeded.");
}
else if (t.Status == TaskStatus.Faulted)
{
    Console.WriteLine("{0} Booster failed", failed);
}

The problem is, I want to limit the number of tasks that run simultaneously. Ideally, I would like to say (this doesn't exist I am just saying what I would like)

maxConcurrentTasks = 4;

I would still like to loop as above and keep adding tasks indefinitely, but I want tasks to block if maxConcurrentTasks is exceeded EVEN IF THERE AREA AVAILABLE cores, and when the number of tasks is below maxConcurrentTasks, any pending tasks are started.

So, if there are 16 symbols, only 4 are being processed at a time. When one is processed, another task that is blocking starts up but the number of running tasks never exceeds maxConcurrentTasks, until the final symbol is processed.

Is there a way to do this?

Ivan
  • 7,448
  • 14
  • 69
  • 134
  • 1
    I answered a similar question here: https://stackoverflow.com/questions/75112527/how-can-i-use-parallel-tasks-and-make-them-not-waiting-each-other?noredirect=1#comment132550179_75112527 – David L Jan 30 '23 at 16:44
  • Here's another dupe-ish answer: https://stackoverflow.com/q/36564596/14357 – spender Jan 30 '23 at 16:46
  • Thaks that's helpful. I know I can queue the tasks myself, but often with these libraries there is already a built in way to do this.... – Ivan Jan 30 '23 at 16:47
  • In essense, I need to throttle myself – Ivan Jan 30 '23 at 16:48
  • @Ivan Yes, being conservative about what you throw at the ThreadPool is definitely a good idea. – spender Jan 30 '23 at 16:51
  • The moment you need to introduce an arbitrary throttle AND independent execution based on that throttle, you're going to need to write something yourself. I'm not aware of anything built-in that solves it for you, when taking async into account. Luckily you can build on top of queues and channels to do so, but you'll have to design the conditions carefully. – David L Jan 30 '23 at 16:51
  • @Ivan `Parallel.ForEach` and `Parallel.ForEachAsync` ([example](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.parallel.foreachasync?view=net-7.0#system-threading-tasks-parallel-foreachasync-1(system-collections-generic-iasyncenumerable((-0))-system-threading-tasks-paralleloptions-system-func((-0-system-threading-cancellationtoken-system-threading-tasks-valuetask))))) accept a [`ParallelOptions`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.paralleloptions?view=net-7.0) as their second (?) parameter. Here you can set `MaxDegreeOfParallelism = 4`. – spender Jan 30 '23 at 16:54

0 Answers0