Task.WhenAll on List behaving differently than Task.WhenAll on IEnumerable

Question

I'm seeing some odd behavioral differences when calling Task.WhenAll(IEnumerable<Task<T>>) and calling Task.WhenAll(List<Task<T>>) while trying to catch exceptions

My code is as follows:

public async Task Run()
{
    var en = GetResources(new []{"a","b","c","d"});
    await foreach (var item in en)
    {
        var res = item.Select(x => x.Id).ToArray();
        System.Console.WriteLine(string.Join("-> ", res));
    }
}

private async IAsyncEnumerable<IEnumerable<ResponseObj>> GetResources(
    IEnumerable<string> identifiers)
{
    IEnumerable<IEnumerable<string>> groupedIds = identifiers.Batch(2);
        // MoreLinq extension method -- batches IEnumerable<T>
        // into IEnumerable<IEnumerable<T>>
    foreach (var batch in groupedIds)
    {
        //GetHttpResource is simply a wrapper around HttpClient which
        //makes an Http request to an API endpoint with the given parameter
        var tasks = batch.Select(id => ac.GetHttpResourceAsync(id)).ToList();
            // if I remove this ToList(), the behavior changes
        var stats = tasks.Select(t => t.Status);
            // at this point the status being WaitingForActivation is reasonable
            // since I have not awaited yet
        IEnumerable<ResponseObj> res = null;
        var taskGroup = Task.WhenAll(tasks);
        try
        {
            res = await taskGroup;
            var awaitedStats = tasks.Select(t => t.Status);
                //this is the part that changes
                //if I have .ToList(), the statuses are RanToCompletion or Faulted
                //if I don't have .ToList(), the statuses are always WaitingForActivation
        }
        catch (Exception ex)
        {
            var exceptions = taskGroup.Exception.InnerException;
            DoSomethingWithExceptions(exceptions);
            res = tasks.Where(g => !g.IsFaulted).Select(t => t.Result);
                //throws an exception because all tasks are WaitingForActivation
        }
        yield return res;
    }
}

Ultimately, I have an IEnumerable of identifiers, I'm batching that into batches of 2 (hard coded in this example), and then running Task.WhenAll to run each batch of 2 at the same time.

What I want is if 1 of the 2 GetResource tasks fails, to still return the successful result of the other, and handle the exception (say, write it to a log).

If I run Task.WhenAll on a list of tasks, this works exactly how I want. However, if I remove the .ToList(), when I attempt to find my faulted tasks in the catch block after the await taskGroup, I run into problems because the statuses of my tasks are still WaitingForActivation although I believe they have been awaited.

When there is no exception thrown, the List and IEnumerable act the same way. This only starts causing issues when I try to catch exceptions.

What is the reasoning behind this behavior? The Task.WhenAll must have completed since I get into the catch block, however why are the statuses still WaitingForActivation? Have I failed to grasp something fundamental here?

Answer 1

Unless you make the list concrete (by using ToList()), each time you enumerate over the list you are calling GetHttpResourceAsync again, and creating a new task. This is due to the deferred execution.

I would definitely keep the ToList() call when working with a list of tasks