2

I have a situation where I am launching Task-s and I want their results to somehow be piped/enqueued in a datastructure as fast as possible, not caring on their order.

Does IAsyncEnumerable fit for this situation?

public async Task BigMethod()
{
    Queue<int> queue = new Queue<int>();
    foreach (var item in RunJobsAsync())
    {
        queue.Enqueue(item);
    }
    //await foreach(var item in await RunIAsyncJobsAsync())
    // {
    //     queue.Enqueue(item);
    // }
    // [some more code]
}

Without IAsyncEnumerable

public async Task<IEnumerable<int> RunJobsAsync()
{    
    List<Task<int>> tasks = new List<Task<int>>();
    foreach(var x in Enumerable.Range(0,100))
    {
        tasks.Add(Task.Run(async()=> await someMethodAsync()));
    }
    await tasks.WhenAll(tasks);
    return tasks.Select(x=>x.Result);
}

With IAsyncEnumerable

public async IAsyncEnumerable<int> RunIAsyncJobsAsync()
{
    foreach (var x in Enumerable.Range(0, 100))
    {
        yield return await Task.Run(async () => await someMethodAsync());
    }
}

Is there any performance gain with IAsyncEnumerable since I want in the end to not move with the algorithm further until all async calls have been finished, but I would like them to execute in parallel, thus instead of waiting all of them sequentially , I'd like to just wait for the longest one to complete.

P.S In this scenario would I need a ConcurrentQueue/locking ?

Peter Csala
  • 17,736
  • 16
  • 35
  • 75
Bercovici Adrian
  • 8,794
  • 17
  • 73
  • 152

1 Answers1

1

No, the IAsyncEnumerable is not a good fit for this case. Since you don't want to move further until all asynchronous operations have been completed, you are not going to take advantage of the IAsyncEnumerables main asset. Which is: streaming the results as soon as they become available. Returning a Task<IEnumerable<int>>, or even better a Task<int[]>, is simpler and preferable. Especially if you also want to launch the operations concurrently, which is something that the IAsyncEnumerable isn't doing by default, and it is quite tricky to achieve.

Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
  • I realized that i do not need to collect the results asap. I think i can stream them all the way. – Bercovici Adrian Oct 28 '20 at 08:19
  • @BercoviciAdrian I hope that you have good reasons to go with the streaming approach, because it will complicate your app considerably. You may have to think about things like buffer capacity, backpressure and such. – Theodor Zoulias Oct 28 '20 at 09:25