Async/Await in foreach with HTTPClient

Question

I have a webservice that loads up some plugins (dlls) and calls their Process method. One of the plugins takes a list of members and ensures that they are all included in a MailChimp list.

Here is the code that adds the users to the MailChimp group.

    private async Task AddMCUsers(List<Member> _memberList)
    {
        using (var http = new HttpClient())
        {
            var creds = Convert.ToBase64String(Encoding.ASCII.GetBytes("user:password");
            http.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", creds);
            string memberURI = string.Format(@"{0}lists/{1}/members", _baseURI, _memberGroupId);
            var jss = new JavaScriptSerializer();

            foreach (var user in _memberlist)
            {
                var _addStatus = "";

                try
                {
                    var content = jss.Serialize(new MCPost()
                    {
                        email_address = user.Email,
                        status = "subscribed",
                        merge_fields = new MCMergeFields()
                        {
                            FNAME = user.Firstname,
                            LNAME = user.Lastname
                        }
                    });

                    using(var result = await http.PostAsync(memberURI, new StringContent(content,Encoding.UTF8, "application/json")))
                    {
                        var resultText = await result.Content.ReadAsStringAsync();

                        if(result.IsSuccessStatusCode)
                        {
                            _addStatus = "Success";
                            var _returnedUser = jss.Deserialize<MCMember>(resultText);
                            //Store new user's id
                            user.ServiceId = _returnedUser.id;
                        }
                        else
                        {
                            _addStatus = "Fail";
                        }
                    }
                }
                catch {
                    _addStatus = "Error";
                }

                LogEvent("Add User - " + _addStatus, string.Format("Id: {0} - {1} {2} (Account: {3}) : {4}", user.Id, user.Firstname, user.Lastname, user.AccountId, user.Email));

            }
        }
    }

In normal procedural code, this wouldn't be a problem. However, the only Post method available on the httpClient was PostAsync. Being fairly new to the async/await stuff, I'm not sure the ramifications on the rest of my code ... particularly as it relates to my attempt to reuse the httpClient instead of instantiating a new one for each http call.

I'm not sure what happens with await when its wrapped in a foreach like I have. Will I run into issues with reusing the httpClient to make repeated calls when running asynchronously?

My other question is, what is actually going to be returned. IOW, my understanding is that await returns a Task. However, here, I'm looping through the list and making multiple calls to await PostAsync. My method returns a Task. But which task gets returned? If my calling method needs to wait for completion before moving on, what does its call look like?

private void Process()
{
    //Get List

    var task = AddMCUsers(list);
    task.Wait();

    //Subsequent processing
 }

I've read that you should use Async all the way. Does this mean my calling method should look more like this?

 public async Task Process()
 {
    //Get list
    ...
    await AddMCUsers(list);

    //Other processing
  }

Thanks to whatever help you can offer on this.

Just a quick suggestion - please don't ever do a `catch { ... }` or `catch (Exception ex) { ... }`. It's just a very bad habit and it will swallow up errors making your code difficult to debug. — Enigmativity, Jun 01 '15 at 23:50

score 4 · Accepted Answer · answered Jun 01 '15 at 23:30

In normal procedural code, this wouldn't be a problem.

The whole point of async/await is to write asynchronous code in a way that looks practically identical to "normal" synchronous code.

Being fairly new to the async/await stuff, I'm not sure the ramifications on the rest of my code ... particularly as it relates to my attempt to reuse the httpClient instead of instantiating a new one for each http call.

HttpClient was intended to be reused; in fact, it can be used for any number of calls simultaneously.

I'm not sure what happens with await when its wrapped in a foreach like I have.

One way to think of it is that await "pauses" the method until its operation completes. When the operation completes, then the method continues executing. I have an async intro that goes into more detail.

Will I run into issues with reusing the httpClient to make repeated calls when running asynchronously?

No, that's fine.

IOW, my understanding is that await returns a Task.

await takes a Task. It "unwraps" that task and returns the result of the task (if any). If the task completed with an exception, then await raises that exception.

My method returns a Task. But which task gets returned?

The Task returned from an async method is created by the async state machine. You don't have to worry about it. See my intro for more details.

If my calling method needs to wait for completion before moving on, what does its call look like? ... I've read that you should use Async all the way. Does this mean my calling method should look more like this?

Yes, it should look like your second snippet:

public async Task ProcessAsync()
{
  //Get list
  ...
  await AddMCUsers(list);

  //Other processing
}

The only thing I changed was the Async suffix, which is recommended by the Task-based Asynchronous Pattern.

Thanks for the answer! One further question, it seems like the way I have things coded, the HTTPClient calls are all waiting on each other rather than running concurrently. IOW, my code -- await http.PostAsync(...) waits on the call before continuing the foreach ... correct? So it can't do further HTTP calls until the first one finishes? Is there a better way to handle that to enable them to run concurrently ... or in parallel -- still having trouble understanding the difference -- running at the same time is what I'm after. — RHarris, Jun 02 '15 at 18:34
@RHarris: Sure! First, factor out the innards of your `foreach` loop into a separate method that adds a single user (`AddMCUserAsync`), and then you can replace your `foreach` loop with a `await Task.WhenAll(_memberlist.Select(x => AddMCUserAsync(x, ...)));` — Stephen Cleary, Jun 02 '15 at 18:38

score 0 · Answer 2 · answered Jun 01 '15 at 22:18

in your code you should be fine with reusing the HttpClient. What async / await is allow the code to release the execution thread to prevent locking a cpu thread while waiting for the web response. It also releases control back to the caller. When releasing code back to the caller it means that if your Process function does not await your AddMCUsers, Process could finish before AddMCUsers (useful in fire and forget situations to not await a method).

What async/await do not do is affect the logical flow of an individual method. When you await an async web call the execution is paused and then resumed at the same point once the web call returns. There is also thread context tracking and the code resumes in the same context (ie. UI thread or background thread depending on the parent) by default, but this can be changed if needed.

At some point in your code you may want to have a method that blocks until your async code competes and that is where you will want your Task.Wait() call to block execution. If all you use is awaits then it is possible for your program to end before your task competes. See the code example below.

class Program
{
    static void Main(string[] args)
    {
        Task waitForMe = Task.Run(() => waitAsync());
    }

    static async Task waitAsync()
    {
        await Task.Delay(5000);
    }
}

in the sample with out a Task.Wait call to block the Main method the program will end before the 5 second wait is complete. Having a main method of the following will cause the program to wait for 5 seconds before exiting:

static void Main(string[] args)
{
    Task waitForMe = Task.Run(() => waitAsync());

    waitForMe.Wait();
}

Async/Await in foreach with HTTPClient

2 Answers2

Linked