Sequential version of Task.WhenAll

Question

Is there a nonblocking Task.WaitAll similar to Task.WhenAll, but not parallel?

I wrote this, but maybe it’s built-in?

public async Task<IEnumerable<T>> AwaitAllAsync<T>(IEnumerable<Task<T>> tasks)
{
    List<T> result = new List<T>();
    foreach(var task in tasks)
    {
        result.Add(await task);
    }

    return result;
}

I want to know if there is a built-in way of waiting for all tasks to complete in async, but a sequential way.

Consider this code:

public class SaveFooCommandHandler : ICommandHandler<SaveFooCommand>
{
   private readonly IBusinessContext context;

   public SaveFooCommandHandler(IBusinessContext context)
   {
      this.context = context;
   }

   public async Task Handle(SaveFooCommand command)
   {
      var foos = (await Task.WhenAll(command.Foos.Select(foo => context.FindAsync<Foo>(foo.Id))).ToList()

      ...
   }
}

That will fail, but

var foos = await context.AwaitAllAsync(command.Foos.Select(foo => context.FindAsync<Foo>(foo.Id));

will not, context.FindAsync is an abstraction of dbcontext.Set<T>().FindAsync

You could do await context.Set<Foo>().Where(f => command.Foos.Contains(f.Id)).ToListAsync(), but the example is simplified.

That doesn't make any sense. `await` doesn't start anything; there isn't anything to _do_ in sequence. — SLaks, May 18 '15 at 14:56
Why do you need a built-in mechanism? Why aren't you using `await Task.WhenAll` which awaits concurrently, not sequentially? — Yuval Itzchakov, May 18 '15 at 14:58
You already know `Task.WhenAll`. I'm sure what your question is. What do you mean by `Task.WhenAll` is paralell? It doesn't do anything in parallel. It just returns a task which completes when all given tasks are completed. — Sriram Sakthivel, May 18 '15 at 15:01
@Anders That's because you're using a single `DbContext`. If you create a new one for each query, you should be fine. — Yuval Itzchakov, May 18 '15 at 15:02
But I do not want to do that, I want the IO to be executed async to free resources but I do not want to spawn a bunch of contexts etc. In this case I'm not interested in parallelism just async exexution — Anders, May 18 '15 at 15:04
*I want the IO to be executed async to free resources but I do not want to spawn a bunch of contacted etc* What is a bunch of contacted? Why do you care if you have multiple instances of `DbContext`? They should be disposed at the end of each query anyway. — Yuval Itzchakov, May 18 '15 at 15:05
@Anders TPL DataFlow enforces message order across the pipeline, which will allow you at the end to wait for all to complete and they will be delivered to the final block in order. What you have shown is pretty much the `WhenAll` implementation, so I'm not sure what you want. After a `WhenAll` call you can simply iterate the collection of tasks you gave to `WhenAll`, which will be the order you denote in your sample code. — Adam Houldsworth, May 18 '15 at 15:10
@AdamHouldsworth: You're completely misunderstanding `Task.WhenAll`. There are no threads involved. Read http://blog.slaks.net/2014-12-23/parallelism-async-threading-explained/ — SLaks, May 18 '15 at 15:18
@SLaks I didn't say there were threads involved with `WhenAll`, in fact I never mention threads at all. I was simply stating that it can be used and still match the sample code that was provided. That said, I very likely misunderstood the question. — Adam Houldsworth, May 18 '15 at 15:22
@AdamHouldsworth: I meant to reply to Anders, not you. (I failed to pay attention to tab completion) — SLaks, May 18 '15 at 15:29
When using when all none thread safe code like EF will fail, so there is definitely not the same as my code — Anders, May 18 '15 at 15:29
@Anders: Then you're doing something else. Show us your code. — SLaks, May 18 '15 at 15:29

score 21 · Accepted Answer · edited Sep 23 '22 at 11:52

I think the core misunderstanding is around the Task type. In asynchronous code, a Task is always already running. So this doesn't make sense:

Is there a non-blocking Task.WaitAll similar to Task.WhenAll but not ~~parallel~~ concurrent?

If you have a collection of tasks, they're all already started.

I want to know if there is a build in way of waiting for all tasks to complete in async but sequential way.

You can, of course, await them sequentially. The standard pattern for this is to use await inside a foreach loop, just like the method you posted.

However, the only reason the sequential-await works is because your LINQ query is lazily evaluated. In particular, if you reify your task collection, it will fail. So this works:

var tasks = command.Foos.Select(foo => context.FindAsync<Foo>(foo.Id));
var foos = await context.AwaitAllAsync(tasks);

and this fails:

var tasks = command.Foos.Select(foo => context.FindAsync<Foo>(foo.Id))
    .ToList();
var foos = await context.AwaitAllAsync(tasks);

Internally, Task.WhenAll reifies your task sequence so it knows how many tasks it needs to wait for.

But this is really beside the point. The real problem you're trying to solve is how to serially execute asynchronous code, which is most easily done using foreach:

var foos = new List<Foo>();
foreach (var fooId in command.Foos.Select(f => f.Id))
  foos.Add(await context.FindAsync<Foo>(fooId));

I was hoping to find a solution which would work with anonymous types. — Mick, Jul 19 '18 at 02:55
How do you declare a variable var foos = new List<> .... I guess you could just have a list of objects but that would make it rather hard to work foos — Mick, Jul 19 '18 at 05:36
@Mick: If you really need a list, then you can derive the type from a generic method, like this: `async Task> MyFunc(this IEnumerable ids, Func> f) { var ret = new List(); foreach (var id in ids) ret.Add(await f(id)); return ret; }`. The hardest part of this is determining what to name `MyFunc`. It should end in `Async` and also convey the meaning that it is a sequential `await`, not a concurrent `await`. — Stephen Cleary, Jul 19 '18 at 13:46
(usage: `var foos = await command.Foos.Select(f => f.Id).MyFunc(id => context.FindAsync(id));` — Stephen Cleary, Jul 19 '18 at 13:48

Sequential version of Task.WhenAll

1 Answers1