launch async methods when others finished, in undetermined order

Question

I have many Tasks, that are running asynchronously

Task<bool> task1 = Task.Run<bool>(() =>
{
    return this.addGroupStringToDictionary("IfcPolyline");
});
Task<bool> task2 = Task.Run<bool>(() =>
{
    return this.addGroupStringToDictionary("IfcPolyLoop");
});
Task<bool> task3 = Task.Run<bool>(() =>
{
    return this.addGroupStringToDictionary("IfcAxis2Placement2D");
});
Task<bool> task4 = Task.Run<bool>(() =>
{
    return this.addGroupStringToDictionary("IfcAxis2Placement3D");
});

Now, I would like to execute other tasks, as soon as some of them finish.

Let's say I have 3 tasks that need to be executed after that :

task5 needs to be executed when Task1 and Task2 finished.
task6 needs to be executed when Task3 and Task4 finished.
task7 needs to be executed when Task1 and Task6 finished.

How can I do that, cause if I use await Task.WhenAll(task1,task2) before calling task5, I also block execution of task6 and task7 ?

score 2 · Accepted Answer · answered Nov 26 '22 at 15:06

2

You could use to your advantage the fact that the Task.Run method accepts both synchronous and asynchronous delegates. Using an asynchronous delegate allows you to await previously started tasks:

var task1 = Task.Run(() => { /*...*/ });

var task2 = Task.Run(() => { /*...*/ });

var task3 = Task.Run(() => { /*...*/ });

var task4 = Task.Run(() => { /*...*/ });

var task5 = Task.Run(async () =>
{
    await Task.WhenAll(task1, task2);
    /*...*/ 
});

var task6 = Task.Run(async () =>
{
    await Task.WhenAll(task3, task4);
    /*...*/ 
});

var task7 = Task.Run(async () =>
{
    await Task.WhenAll(task1, task6);
    /*...*/ 
});

answered Nov 26 '22 at 15:06

Theodor Zoulias

34,835
7
69
104

It works perfectly, but it seems that system doesn't like when too much async methods work at the same time? On some other questions for .NET they are talking about thousands of parallel tasks, and I have maximum 150 tasks (well, 30 for now). – Siegfried.V Nov 27 '22 at 08:38
@Siegfried.V the `Task.Run` schedules the work on the `ThreadPool`, and the `ThreadPool` becomes saturated when too much work is scheduled on it. You can learn more about the behavior of the `ThreadPool`, and how to affect it, [here](https://stackoverflow.com/questions/73328524/are-there-more-tasks-being-performed-than-threads-whats-happening/73328718#73328718). You could also consider switching to native asynchronous APIs that do work [without blocking threads](https://blog.stephencleary.com/2013/11/there-is-no-thread.html), in case such APIs are available for whatever you are doing. – Theodor Zoulias Nov 27 '22 at 12:37
The problem is that each task works on the same dictionaries, I made a post about it today `https://stackoverflow.com/questions/74589066/executing-same-function-from-several-async-methods-simultenously-causes-errors`. I still need to learn a lot about async I guess... – Siegfried.V Nov 27 '22 at 13:03
@Siegfried.V yep, you need to know about [thread-safety](https://www.albahari.com/threading/part2.aspx#_Thread_Safety "Threading in C# - BASIC SYNCHRONIZATION - Thread Safety") when you are doing multithreading. From my experience the question that you have asked in this post doesn't occur frequently in practice. So I guess that there are better ways to do whatever you are trying to do here. You'll learn these ways as you become more experienced. – Theodor Zoulias Nov 27 '22 at 13:17
1

in fact, I did use Concurrent dictionary, and all worked fine, but on the end, It takes as much time as if I did it without async metods, so for now I will stay without async, and wait to master them better before trying to optimize my code. There is a way to do better (don't use a List, but create a Dictionary independant for each class. But as I have 120 classes, it will take a bit of time to create them all, and I am short in time, so will be for later... – Siegfried.V Nov 27 '22 at 13:24
1

@Siegfried.V I have posted some thoughts about the `ConcurrentDictionary` [here](https://stackoverflow.com/questions/42013226/when-should-i-use-concurrentdictionary-and-dictionary/63940194#63940194). – Theodor Zoulias Nov 27 '22 at 13:32

score 1 · Answer 2 · answered Nov 26 '22 at 14:15

1

For a simple case, you can use Task.ContinueWith(). See Chaining tasks using continuation tasks on Microsoft Learn

For more complex case if you have lots of data to process, you can use DataFlow library to create a pipeline for asynchronous jobs. See How to: Implement a producer-consumer dataflow pattern

answered Nov 26 '22 at 14:15

user700390

2,287
1
19
26

Well, as I read, there is no "magical way", kind of 'Task task3 = Task.RunWhenAll(Task1, Task2)`... cause the `Task.ContinueWith()` can be used on only one task? I guess I need to look forward on your second link, but will be a bit longer to learn. – Siegfried.V Nov 26 '22 at 14:30

launch async methods when others finished, in undetermined order

2 Answers2