Will the following snipped be awaited once or on each step?

Question

I have the following code snipped and I'm not really sure how it will be processed. My intention here is to await all tasks at once, but I still want to utilize async disposal logic

var copyTasks = attachmentsDict.Select(async x =>
{
    var (file, document) = x;
    var path = BuildFilePath(document.Name);
    var directory = Path.GetDirectoryName(path);

    if (!Directory.Exists(directory))
    {
        Directory.CreateDirectory(directory!);
    }

    await using var fs = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None, 4096, true);
    await file.Stream.CopyToAsync(fs, cancellationToken);
}).ToList();

await Task.WhenAll(copyTasks);

Should I keep it like this? Is it better to rewrite it into processing files one by one?

only thing I would change is calling `ConfigureAwait` - https://stackoverflow.com/questions/27851073 — Rand Random, Jun 07 '23 at 13:23
Relevant: https://stackoverflow.com/a/35016869/27678 ("Async await in linq select") — AndyG, Jun 07 '23 at 13:23
"Better" is a little too subjective for StackOverflow, but my 2 cents is that you will probably oversaturate your I/O devices by doing them all at once, resulting in blocking (especially if the files are large). It's probably efficient to do a few at once, but the sweet spot is specific to your specific hardware and workload. — AndyG, Jun 07 '23 at 13:29
@RandRandom It is inside of ASP.Core project without SynchronizationContext, but thanks for checking it :) — Renat Zamaletdinov, Jun 07 '23 at 13:30
@AndyG If I understood that link correctly, my snippet will be awaited only once, so now I know. But still, your point is more than valid, and most likely I will rewrite it into sequential processing — Renat Zamaletdinov, Jun 07 '23 at 14:18
If you want sequential consider using https://learn.microsoft.com/dotnet/api/system.threading.tasks.parallel.foreachasync — Rand Random, Jun 07 '23 at 18:41
Throw in a `SemaphoreSlim` and you can easily limit how much work you want to run in parallel, and where in the method you will pause to wait for that. — Jeremy Lakeman, Jun 27 '23 at 01:04

score 0 · Answer 1 · answered Jun 26 '23 at 22:26

0

Asynchrony will not have valuable positive (parallel processing) effect if you use same physical drive for your attachment files. If so - better to be honest and process files asynchronously, but work synchronously with collection (by using foreach loop instead of .Select(async x => {...})

answered Jun 26 '23 at 22:26

David Levin

6,573
5
48
80

Blocking for each file close, before calling the next file open, can have a significant impact on execution time. If you are ever processing a large number of files, opening files ahead of time is the first optimisation I would do. – Jeremy Lakeman Jun 27 '23 at 01:06

Will the following snipped be awaited once or on each step?

1 Answers1