I am using Task classes to do multi-threaded computation.
For example:
List<Task> taskList = new List<Task>();
for(int i=0;i<10;i++){
var task = new Task(() => Console.WriteLine("Hello from taskA."));
taskList.Add(task);
task.Start();
}
Is there a way to make only, let's say, 3 tasks to run at most, and the rest to wait?
My blog post shows how to do this both with Tasks and with Actions, and provides a sample project you can download and run to see both in action.
If using Actions, you can use the built-in .Net Parallel.Invoke function. Here we limit it to running at most 3 threads in parallel.
var listOfActions = new List<Action>();
for (int i = 0; i < 10; i++)
{
// Note that we create the Action here, but do not start it.
listOfActions.Add(() => DoSomething());
}
var options = new ParallelOptions {MaxDegreeOfParallelism = 3};
Parallel.Invoke(options, listOfActions.ToArray());
Since you are using Tasks here though, there is no built-in function. However, you can use the one that I provide on my blog.
/// <summary>
/// Starts the given tasks and waits for them to complete. This will run, at most, the specified number of tasks in parallel.
/// <para>NOTE: If one of the given tasks has already been started, an exception will be thrown.</para>
/// </summary>
/// <param name="tasksToRun">The tasks to run.</param>
/// <param name="maxTasksToRunInParallel">The maximum number of tasks to run in parallel.</param>
/// <param name="cancellationToken">The cancellation token.</param>
public static void StartAndWaitAllThrottled(IEnumerable<Task> tasksToRun, int maxTasksToRunInParallel, CancellationToken cancellationToken = new CancellationToken())
{
StartAndWaitAllThrottled(tasksToRun, maxTasksToRunInParallel, -1, cancellationToken);
}
/// <summary>
/// Starts the given tasks and waits for them to complete. This will run, at most, the specified number of tasks in parallel.
/// <para>NOTE: If one of the given tasks has already been started, an exception will be thrown.</para>
/// </summary>
/// <param name="tasksToRun">The tasks to run.</param>
/// <param name="maxTasksToRunInParallel">The maximum number of tasks to run in parallel.</param>
/// <param name="timeoutInMilliseconds">The maximum milliseconds we should allow the max tasks to run in parallel before allowing another task to start. Specify -1 to wait indefinitely.</param>
/// <param name="cancellationToken">The cancellation token.</param>
public static void StartAndWaitAllThrottled(IEnumerable<Task> tasksToRun, int maxTasksToRunInParallel, int timeoutInMilliseconds, CancellationToken cancellationToken = new CancellationToken())
{
// Convert to a list of tasks so that we don't enumerate over it multiple times needlessly.
var tasks = tasksToRun.ToList();
using (var throttler = new SemaphoreSlim(maxTasksToRunInParallel))
{
var postTaskTasks = new List<Task>();
// Have each task notify the throttler when it completes so that it decrements the number of tasks currently running.
tasks.ForEach(t => postTaskTasks.Add(t.ContinueWith(tsk => throttler.Release())));
// Start running each task.
foreach (var task in tasks)
{
// Increment the number of tasks currently running and wait if too many are running.
throttler.Wait(timeoutInMilliseconds, cancellationToken);
cancellationToken.ThrowIfCancellationRequested();
task.Start();
}
// Wait for all of the provided tasks to complete.
// We wait on the list of "post" tasks instead of the original tasks, otherwise there is a potential race condition where the throttler's using block is exited before some Tasks have had their "post" action completed, which references the throttler, resulting in an exception due to accessing a disposed object.
Task.WaitAll(postTaskTasks.ToArray(), cancellationToken);
}
}
And then creating your list of Tasks and calling the function to have them run, with say a maximum of 3 simultaneous at a time, you could do this:
var listOfTasks = new List<Task>();
for (int i = 0; i < 10; i++)
{
var count = i;
// Note that we create the Task here, but do not start it.
listOfTasks.Add(new Task(() => Something()));
}
Tasks.StartAndWaitAllThrottled(listOfTasks, 3);
I recommend checking the post here.
To paraphrase, you create a collection of the relevant tasks, then use the ParallelOptions.MaxDegreeOfParallelism to control how many get executed at the same time.
You can use this one as a generic
public TaskThrottle(int initialCount, int maxTasksToRunInParallel)
{
_semaphore = new SemaphoreSlim(initialCount, maxTasksToRunInParallel);
}
public void TaskThrottler<T>(IEnumerable<Task<T>> tasks, int timeoutInMilliseconds, CancellationToken cancellationToken = default(CancellationToken)) where T : class
{
// Get Tasks as List
var taskList = tasks as IList<Task<T>> ?? tasks.ToList();
var semaphoreTasks = new List<Task<int>>();
// When the first task completed, flag as done/release
taskList.ForEach(x =>
{
semaphoreTasks.Add(x.ContinueWith(y => _semaphore.Release(), cancellationToken));
});
semaphoreTasks.ForEach(async x =>
{
// It will not pass this until one free slot available or timeout occure
if(timeoutInMilliseconds > 0)
await _semaphore.WaitAsync(timeoutInMilliseconds, cancellationToken);
else
await _semaphore.WaitAsync(cancellationToken);
// Throws a OperationCanceledException if this token has had cancellation requested
cancellationToken.ThrowIfCancellationRequested();
// Start the task
x.Start();
});
Task.WaitAll(semaphoreTasks.ToArray(), cancellationToken);
}
