Returning IEnumerable from yield in C#

Question

I am able to compile

public async Task<IEnumerable<T>> GetResultsFromQueryExecutionId<T>(string queryExecutionId)
{
     await using var csvResponseStream = await transferUtility.OpenStreamAsync("bucket", "blah.csv");
     return GetResultsFromResponseStream<T>(csvResponseStream);
}

private IEnumerable<T> GetResultsFromResponseStream<T>(Stream csvResponseStream)
{
     using var streamReader = new StreamReader(csvResponseStream);
     using var csvReader = new CsvReader(streamReader, csvConfiguration);
     foreach (var record in csvReader.GetRecords<T>())
     {
          yield return record;
     }
}

but if I try to remove the private method and run the code in one method I get a compile error "The body of 'GetResultsFromQueryExecutionId' cannot be an iterator block because 'System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable>' is not an async iterator interface type"

I tried

public async Task<IEnumerable<T>> GetResultsFromQueryExecutionId<T>(string queryExecutionId, string logPrefix = null)
{
     await using var csvResponseStream = await transferUtility.OpenStreamAsync("bucket", "blah.csv");
     using var streamReader = new StreamReader(csvResponseStream);
     using var csvReader = new CsvReader(streamReader, csvConfiguration);
     foreach (var record in csvReader.GetRecords<T>())
     {
          yield return record;
     }
}

I was expecting the code to be equivalent. Is anyone able to explain why the compiler will not allow me to combine this into one method please?

Answer 1

As written in the specification:

The yield statement is used in an iterator block (§13.3) to yield a value to the enumerator object (§15.14.5) or enumerable object (§15.14.6) of an iterator or to signal the end of the iteration.

Currently as far as I know compiler supports returning the following types from methods using yield return:

IEnumerable
IEnumerator
IEnumerable<T>
IEnumerator<T>
IAsyncEnumerable<T>
IAsyncEnumerator<T>

And Task<IEnumerable<T>> is not one of them, so you are limited to either using approach with two methods or using IAsyncEnumerable/IAsyncEnumerator:

private async IAsyncEnumerable<T> GetResultsFromResponseStream<T>(Stream csvResponseStream)
{
    await using var csvResponseStream = ... 
    // the iterator block
}

Though personally I would use the first one (you can turn the method into a local function for convenience).

Read more:

• Iterating with Async Enumerables in C# 8
• yield statement - provide the next element

Answer 2

yield return is a very special construct that forces the method it's in to get compiled in a completely different mode. You can only return IAsyncEnumerable<> or IEnumerable<> (or types with similar methods on them) from methods that use it, so if you want to return something different you'll need to break the code into separate methods as you have.

That said, you may want to look at a couple of other options. You can use a Local Function that's nested inside of your outer function, to help keep things grouped together.

public async Task<IEnumerable<T>> GetResultsFromQueryExecutionId<T>(string queryExecutionId)
{
    await Task.Yield();
    return GetResultsFromResponseStream();
    IEnumerable<T> GetResultsFromResponseStream()
    {
        foreach (var record in new T[0] )
        {
            yield return record;
        }
    }
}

It might be appropriate to return an IAsyncEnumerable<T> instead:

public async IAsyncEnumerable<T> GetResultsFromQueryExecutionId<T>(string queryExecutionId)
{
    await Task.Yield();
    foreach (var record in new T[0] )
    {
        yield return record;
    }
}

This would change the way people use your method.

// Before
var results = await GetResultsFromQueryExecutionId<int>("foo");
foreach(var result in results)
{
    ...
}
// After 
var results = GetResultsFromQueryExecutionId<int>("foo");
await foreach (var result in results)
{
    ...
}