Asynchronous requests to API, using yield iterator?

Question

I have some trouble choosing the best way to go with asynchronous requests to a web API. I need to get data from sequential webrequests, with the following requirements :

requests must obviously not block GUI (async or in another thread)
each request returns a list of records, that I want to merge immediately with my app's data
requests must not run at the same time (preserve "logical" order)
if a request fails, remaining requests must be cancelled

The web requests are not CPU intensive, just slow. Moreover, since I want to merge results after every request, using different threads would cause some synchronization overhead.

That's why I thought better to use async/await to avoid creating new threads. First, is this assumption right ?

My (incomplete) solution :

I came up with an elegant solution using yield, but it doesn't implement the expected async behavior.

Here is the simplified API's connector code :

public IEnumerable<Record[]> GetRecordsInRange(DateTime startDate, DateTime endDate)
{
    while (startDate <= endDate)
    {
        List<Record> results = new List<Record>();

        try
        {
            results.AddRange(ExecuteQueryAPI(startDate, QueryTypes.A));
            results.AddRange(ExecuteQueryAPI(startDate, QueryTypes.B));
            results.AddRange(ExecuteQueryAPI(startDate, QueryTypes.C));
        }
        catch (Exception ex)
        {
            // Cancel remaining requests
            yield break;
        }

        // Return partial results.
        yield return results.ToArray();

        // Jump to the next day.
        startDate = startDate.AddDays(1);
    }

    yield break;
}

Usage :

foreach (var records in APIConnector.GetRecordsInRange(DateTime.Now.AddDays(-10), DateTime.Now))
{
    Records.AddRange(records);
}

Is it possible to implement an aync/await behavior to the previous code, or should I use a different approach ?

Thanks, I'll take a close look to it. A subsidiary question would be: should I implement this kind of solution, or is there a better solution ? — Profet, Dec 12 '16 at 12:43
Maybe this? http://stackoverflow.com/q/23295119/993547 using Rx. — Patrick Hofman, Dec 12 '16 at 12:44

score 2 · Answer 1 · answered Dec 12 '16 at 14:21

async-await is good enough approach in your case.
This approach will use only one thread and will not block UI.

public async Task<IEnumerable<Record>> GetRecordsInRange(DateTime startDate, 
                                                         DateTime endDate)
{
    while (startDate <= endDate)
    {
        List<Record> results = new List<Record>();
        // send requests in order you want
        var taskA = ExecuteQueryAPIAsync(startDate, QueryTypes.A);
        var taskB = ExecuteQueryAPIAsync(startDate, QueryTypes.B);
        var taskC = ExecuteQueryAPIAsync(startDate, QueryTypes.C);

        // observes results in same order
        var resultA = await taskA;
        results.AddRange(resultA);

        var resultB = await taskB;
        results.AddRange(resultB);

        var resultC = await taskC;
        results.AddRange(resultC);

        // Jump to the next day.
        startDate = startDate.AddDays(1);
    }

    return results;
}

You need create ExecuteQueryAPIAsync method which will sget response asynchronously.

await taskA keyword will return collection of results or throw exception.

K Ekegren · Answer 2 · 2016-12-12T13:10:05.790

A problem I see with your code is that you run three api requests for each yield, making each iteration three times as slow as it has to. I would start by rewrite it to something like this:

    public IEnumerable<Record[]> GetRecordsInRange(DateTime startDate, DateTime endDate)
    {
        while(startDate <= endDate)
        {
            // This hard-coded stuff should be replaced with something prettier but that's off topic and comes down to your implementation and the meaning of QueryTypes
            // I choose to isolate it to the delegates variable to make it easy to rewrite and perhaps instantiate it from a function
            var delegates = new Func<Record[]>[] 
            {
                ()=>ExecuteQueryAPI(startDate, QueryTypes.A),
                ()=>ExecuteQueryAPI(startDate, QueryTypes.B),
                ()=>ExecuteQueryAPI(startDate, QueryTypes.C)
            };

            foreach(var d in delegates)
            {
                Record[] output;
                try
                {
                    output = d();
                }
                catch
                {
                    yield break;
                }
                yield return output;
            }

            startDate = startDate.AddDays(1);
        }
    }

This would make the code run properly and it should be fairly simple to run it async

You right, but this was by design! Because my results must include Type A, B and C results or nothing :) — Profet, Dec 12 '16 at 13:09

Asynchronous requests to API, using yield iterator?

My (incomplete) solution :

2 Answers2