`WaitAll()` or `WhenAll` when expecting data

Question

I've never attempted to use WaitAll() or WhenAll() when running async functionality. After looking at many documentations, SO posts, and tutorials, I haven't found enough information for this, so here I am.

I'm trying to figure out the best/proper way(s) to do the following:

Using EF6, get data as List<Entity>. Iterate through each Entity and call an external API to perform some action. External API returns data per Entity which I need to store on the same Entity.

Currently I have built (not tested) the following (without the error handling code):

public IEnumerable<Entity> Process() {
    bool hasChanged = false;
    var data = _db.Entity.Where(x => !x.IsRegistered);
    
    foreach (var entity in data) {
        var result = await CallExternalApi(entity.Id, entity.Name);

        entity.RegistrationId = result.RegistrationId;
        entity.IsRegistered = true;
        _db.Entry(entity).State = EntityState.Modified;
        hasChanges = true;
    }

    if (hasChanges) {
        uow.Commit();
    }

    return data;
}

I feel like I may be able to take advantage of some other functionality/feature in async, but if I can I'm not sure how to implement it here.

Any guidance is really appreciated.

Update

The API I'm calling is the Zoom Api to add Registrants. While they do have an route to batch add Registrants, it does not return the RegistrantId and the Join Url I need.

Answer 1

First, figure out if your external API might have a way to get all the items you want in a batch. If it does, use that instead of sending a whole bunch of requests.

If you need to send a separate request for each item, but want to do it concurrently, you could do this:

public async Task<IReadOnlyCollection<Entity>> Process() {
    var data = _db.Entity.Where(x => !x.IsRegistered).ToList();

    if(!data.Any()) { return data; }

    var entityResultTasks = data
        .Select(async entity => new { entity, result = await CallExternalApi(entity.Id, entity.Name) })
        .ToList();
    var entityResults = await Task.WhenAll(entityResultTasks);
    foreach (var entityResult in entityResults) {
        var entity = entityResult.entity;
        var result = entityResult.result;

        entity.RegistrationId = result.RegistrationId;
        entity.IsRegistered = true;
        _db.Entry(entity).State = EntityState.Modified;
    }
    uow.Commit();
    return data;
}

You will want to watch out for possible concurrency limits on the target source. Consider using Chunk to break your work into batches of acceptable sizes, or leveraging a semaphore or something to throttle the number of calls you're making.