How to make Parallel wait until all async task finished?

Question

I am making a resource loading code. But, there is a problem.

var toDoList = new List<Action>();

toDoList.Add(async () =>
{
  await CanvasBitmap.LoadAsync(PATH1);
  Console.WriteLine("Load image");
});
toDoList.Add(async () =>
{
  await CanvasBitmap.LoadAsync(PATH2);
  Console.WriteLine("Load image");
});

// ...

toDoList.Add(async () =>
{
  await CanvasBitmap.LoadAsync(PATH100);
  Console.WriteLine("Load image");
});

toDoList.Add(() =>
{
  AudioSystem.Instance.Load(AUDIO_PATH1);
  Console.WriteLine("Load audio");
});

toDoList.Add(() =>
{
  AudioSystem.Instance.Load(AUDIO_PATH2);
  Console.WriteLine("Load audio");
});

// ...

toDoList.Add(() =>
{
  AudioSystem.Instance.Load(AUDIO_PATH100);
  Console.WriteLine("Load audio");
});

Console.WriteLine("Parallel Load Start");
Parallel.ForEach(toDoList, toDo => {
  toDo();
});
Console.WriteLine("Parallel Load End");

I want to make Parallel wait until all LoadAsync function finished. But, because of async Action delegate, I didn't wait. It shows that

Parallel Load Start
Load audio
Load audio
Load image
Load audio
Load image
...
Load audio
Parallel Load End
Load image
Load image
Load image
...

There is no method to load image synchronously because it is a part of Win2D. (= WinRT API)
I understand why all disk I/O task must be async method.
But, above code is running at not UI thread but custom thread. So, it doesn't need to run async.
You can think that Parallel is useless due to async method. But, as you can see, toDoValues has both async (Win2D), sync (Audio). Parallel is needed for loading audio file effectively.

How can I make Parallel wait until all resources are loaded?

Yeah you are coming at this from the wrong angle, jsut used tasks, and wait them all using `WhenAll` or `WaitAll` — TheGeneral, Jan 29 '19 at 11:11
Your problem is not connected with Parallel. Parallel waits untill all of the Actions are executed. You start async Tasks and don't wait until they are done. — Access Denied, Jan 29 '19 at 11:13
Some of your actions are themselves `async`, because no `Task` is returned you have "fire-and-forget" actions. The action will complete when the task has been started, not when it completes. Such actions are not directly suitable to be scheduled alongside synchronous actions: either have all `Func<..., Task>` or all synchronous. — Richard, Jan 29 '19 at 11:13
I understood why it occured. I knew that Parallel is waiting all tasks. But, the tasks is async method. Async method returned as soon as it called. So, Parallel recognized that all tasks returned and it finished. So, I used a SemaphoreSlim (init 0, max 1). and Wait after Parallel, and Wake up when all resources are loaded. It works well. But, code seems to be dirty. — P. ful, Jan 29 '19 at 11:50
I post this question because Win2D LoadAsync method doesn't have any synchronous method. I tried LoadAsync -> Wait -> Result yesterday. But, it throws ArgumentException. But, When I tried that again just now, it works! ??? So, it was solved. Thanks for all comments! — P. ful, Jan 29 '19 at 17:04

score 3 · Accepted Answer · answered Jan 29 '19 at 14:43

Parallel is needed for loading audio file effectively.

Well, Parallel is one approach to doing parallel operations. But your code is currently mixing CPU-bound operations and asynchronous operations and trying to combine them using Parallel, which is only for CPU-bound operations. That's why it's not working well.

A better approach would be to push the CPU-bound operations onto the thread pool, using Task.Run rather than Parallel. Then you can treat them as asynchronous (from the UI thread's perspective), and combine them with the naturally-asynchronous I/O operations using Task.WhenAll.

Using your example code, it would look like this:

var toDoList = new List<Func<Task>>();
toDoList.Add(async () =>
{
  await CanvasBitmap.LoadAsync(PATH1);
  Console.WriteLine("Load image");
});
toDoList.Add(async () =>
{
  await CanvasBitmap.LoadAsync(PATH2);
  Console.WriteLine("Load image");
});

// ...

toDoList.Add(async () =>
{
  await CanvasBitmap.LoadAsync(PATH100);
  Console.WriteLine("Load image");
});

toDoList.Add(() => Task.Run(() =>
{
  AudioSystem.Instance.Load(AUDIO_PATH1);
  Console.WriteLine("Load audio");
}));

toDoList.Add(() => Task.Run(() =>
{
  AudioSystem.Instance.Load(AUDIO_PATH2);
  Console.WriteLine("Load audio");
}));

// ...

toDoList.Add(() => Task.Run(() =>
{
  AudioSystem.Instance.Load(AUDIO_PATH100);
  Console.WriteLine("Load audio");
}));

Console.WriteLine("Concurrent Load Start");
var tasks = toDoList.Select(toDo => toDo()).ToList();
Console.WriteLine("Concurrent Load Started");
await Task.WhenAll(tasks);
Console.WriteLine("Concurrent Load End");

Thanks a lot! It works! But, when I want to cancel this task, how can I cancel tasks? When I use Parallel, CancellationToken is useful. But, in image loading async method, there is no space to insert CancellationToken. — P. ful, Jan 29 '19 at 16:42
@P.ful: You can pass the `CancellationToken` into your delegates and use `ThrowIfCancellationRequested`. You might also want to limit the number of tasks running at a time using `SemaphoreSlim`. — Stephen Cleary, Jan 29 '19 at 18:37

score 1 · Answer 2 · answered Jan 29 '19 at 14:06

1

You need to use WhenAll.

Console.WriteLine("Parallel Load Start");
Parallel.ForEach(toDoList, toDo => {
  toDo();
});

await Task.WhenAll(toDoList); // Next line will be executed when all tasks are completed,

Console.WriteLine("Parallel Load End");

answered Jan 29 '19 at 14:06

fhnaseer

7,159
16
60
112

Thanks! I will try it. – P. ful Jan 29 '19 at 16:42

How to make Parallel wait until all async task finished?

2 Answers2