How to run batch of Tasks?

Question

I have a lot of Task what i running simultaneously. But sometimes there are too much of them to run in same time. So i want to run them by bunches for 100 Tasks. But i'm not sure how can i modify my code.
There is my current code:

protected void ValidateFile(List<MyFile> validFiles, MyFile file)
{
    ///do something
    validFiles.add(file);
}

internal Task ValidateFilesAsync(List<MyFile> validFiles, SplashScreenManager splashScreen, MyFile file)
{
    return Task.Run(() => ValidateFile(validFiles, file)).ContinueWith(
        t => splashScreen?.SendCommand(SplashScreen.SplashScreenCommand.IncreaseGeneralActionValue,
            1));
}

var validFiles = new List<MyFile>;
var tasks = new List<Task>();
foreach (var file in filesToValidate)
{
    tasks.Add(ValidateFilesAsync(validFiles, splashScreenManager, file));
}
Task.WaitAll(tasks.ToArray());

I'm not very good with Tasks so code may be not an optimal, but somehow it's working.
I found that i can use Parallel.Foreach with MaxDegreeOfParallelism param, but for this i have to ValidateFile into Action and remove ValidateFilesAsync and in this case i will lose ContinueWith functional what i use to increase progress bar in gui.
How can i achive restriction of simultaneously running Tasks and if possible save ContinueWith-like functional?

Does this answer your question? [Have a set of Tasks with only X running at a time](https://stackoverflow.com/questions/14075029/have-a-set-of-tasks-with-only-x-running-at-a-time) — Sinatr, May 20 '21 at 09:25
@Sinatr in this case i don't need to use `ValidateFilesAsync` and should run the `ValidateFile`? — Kliver Max, May 20 '21 at 09:35
Have a look into [DataFlow](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library). This walkthroug may be interesting for your usecase [Walkthrough: Using BatchBlock and BatchedJoinBlock to Improve Efficiency](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/walkthrough-using-batchblock-and-batchedjoinblock-to-improve-efficiency) — Fildor, May 20 '21 at 09:42

score 0 · Answer 1 · answered May 20 '21 at 09:43

I would suggest using Parallel.Foreach. Not sure what you mean with " have to ValidateFile into Action", just change your foreach-body to a lambda. There should be plenty of examples easily available.

To update the UI there are several ways to do it:

Use SynchronizationContext
Start a task using a task scheduler that runs the task on the UI thread
Update the progress variable on the background thread, and use a timer to poll the progress variable from the UI thread.

Keep in mind that IO, like reading files, may not improve much, if at all, by reading in parallel. Spinning disks are inherently serial, and have a fairly large seek times. SSDs are inherently parallel, but I would still not expect any huge performance gains from reading in parallel, especially not going up to 100 concurrent reads.

How to run batch of Tasks?

1 Answers1