Task.WhenAll but process results one by one

Question

Let's say I have a list of Tasks, and I want to run them in parallel. But I don't need all of them to finish to continue, I can move on with just one. The following code waits for all the tasks to finish to move on. Is there a way for me to individually continue with the task that has completed while waiting for the other ones to finish?

List<string>[] x = await Task.WhenAll(new Task<List<string>>[] { task1, task2 })
// When task1 finishes, I want to process the result immediately
// instead of waiting on task2.

Answer 1

You're probably looking for Task.WhenAny.

I've used it for setting off a pile of tasks and then processing each of them as they become ready, but I suppose you could also just wait for one to finish and continue without the loop if you don't care about dealing with the rest.

while (tasks.Count() > 0)
{
    var task = await Task.WhenAny(tasks);
    tasks.Remove(task);
    var taskresult = await task;
    // process result
}

Answer 2

If you are using C# 8 and .NET Core you can take advantage of IAsyncEnumerable to hide this complexity from the consuming side.

Just like this:

static async Task Main(string[] args)
{
    await foreach (var data in GetData())
    {
        Console.WriteLine(data);
    }

    Console.ReadLine();
}

static async IAsyncEnumerable<string> GetData()
{
    List<Task<string>> tasks = new List<Task<string>> {GetData1(), GetData3(), GetData2()};

    while (tasks.Any())
    {
        var finishedTask = await Task.WhenAny(tasks);
        tasks.Remove(finishedTask);
        yield return await finishedTask;
    }
}

static async Task<string> GetData1()
{
    await Task.Delay(5000);
    return "Data1";
}

static async Task<string> GetData2()
{
    await Task.Delay(3000);
    return "Data2";
}

static async Task<string> GetData3()
{
    await Task.Delay(2000);
    return "Data3";
}

Answer 3

You can use Task.WhenAny instead.

Example "stolen" from Stephen Cleary's Blog:

var client = new HttpClient();
string results = await await Task.WhenAny(
    client.GetStringAsync("http://example.com"),
    client.GetStringAsync("http://microsoft.com"));
// results contains the HTML for whichever website responded first.

Responding to comment

You absolutely can keep track of the other tasks:

// supposing you have a list of Tasks in `myTasks`:
while (myTasks.Count > 0)
{
    var finishedTask = await Task.WhenAny(myTasks);
    myTasks.Remove(finishedTask);
    handleFinishedTask(finishedTask); // assuming this is a method that
                                      // does the work on finished tasks.
}

The only thing you'd have to watch out for is:

The returned task will always end in the RanToCompletion state with its Result set to the first task to complete. This is true even if the first task to complete ended in the Canceled or Faulted state.

Remarks in Task.WhenAny docs (emphasis by me).

Answer 4

In case you want to process the results in order of the completion of the tasks, there is the OrderByCompletion extension method that does exactly that in Stephen Cleary's Nito.AsyncEx library, with the signature below:

// Creates a new collection of tasks that complete in order.
public static List<Task<T>> OrderByCompletion<T>(this IEnumerable<Task<T>> @this);

Usage example:

Task<string>[] tasks = new[] { task1, task2, task3, task4 };
foreach (var task in tasks.OrderByCompletion())
{
    string result = await task;
    // Do something with result
}

Answer 5

Based on the answer of Peter Csala, here a extension method for IAsyncEnumerable:

    public static async IAsyncEnumerable<T> OrderedByCompletion<T>(this IEnumerable<Task<T>> tasks)
    {
        List<Task<T>> taskList = new List<Task<T>>(tasks);

        while (taskList.Count > 0)
        {
            var finishedTask = await Task.WhenAny(taskList);
            taskList.Remove(finishedTask);
            yield return await finishedTask;
        }
    }