Running Async Foreach Loop C# async await

Question

I am struggling to grasp the basic concept of c# async await.

Basically what I have is a List of objects which I need to process, the processing involves iterating through its properties and joining strings, and then creating a new object (in this case called a trellocard) and eventually adding a list of trellocards.

The iteration takes quiet a long time, So what I would like to do is process multiple objects at asynchronously.

I've tried multiple approaches but basically I want to do something like this. (in the below example I have removed the processing, and just put system.threading.thread.sleep(200). Im await that this is NOT an async method, and I could use tasks.delay but the point is my processing does not have any async methods, i want to just run the entire method with multiple instances.

    private async Task<List<TrelloCard>> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
    {
        List<TrelloCard> cards = new List<TrelloCard>();

        foreach (var job in jobs.ToList())
        {
           card = await ProcessCards(job, cards);  // I would like to run multiple instances of the processing
           cards.add(card); //Once each instance is finshed it adds it to the list
        }


  private async Task<TrelloCard> ProcessCards(Job job)
    {
        System.Threading.Thread.Sleep(2000);  //Just for examples sake

        return new TrelloCard();
    }

score 8 · Accepted Answer · edited Apr 13 '17 at 12:40

I am struggling to grasp the basic concept of c# async await.

Simple definition would be, Async-Await is a part .Net concurrency, which can be used to make multiple IO calls, and in process not waste the Threads, which are meant for Compute operations. Its like call to Database, Web service, Network calls, File IO, all of which doesn't need a current process thread

In your current case, where the use case is:

iterating through its properties and joining strings, and then creating a new object
eventually adding a list of trellocards

This seems to be a compute bound operation, until and unless you are doing an IO, to me it seems you are traversing an in memory object, for this case the better choice would be:

Parallel.ForEach, to parallelize the in memory processing, though you need to be careful of Race conditions, as a given memory could be accessed by multiple threads, thus corrupting it specially during write operation, so at least in current code use Thread safe collection like ConcurrentBag from System.Collections.Concurrent namespace, or which ever suit the use case instead of List<TrelloCard>, or you may consider following Thread safe list

Also please note that, in case your methods are not by default Async, then you may plan to wrap them in a Task.Run, to await upon, though this would need a Thread pool thread, but can be called using Async-Await

Parallel.Foreach code for your use case (I am doing direct replacement, there seems to be an issue in your code, since ProcessCards function, just takes Job object but you are also passing the collection Cards, which is compilation error):

 private List<TrelloCard> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
    {
        ConcurrentBag<TrelloCard> cards = new ConcurrentBag<TrelloCard>();

        Parallel.ForEach(jobs.ToList(), (job) => 
        {
           card = ProcessCards(job);  // I would like to run multiple instances of the processing
           cards.Add(card); //Once each instance is finshed it adds it to the list
        });

           return cards.ToList();
   }

  private TrelloCard ProcessCards(Job job)
    {
       return new TrelloCard();
    }

thank you!. Its amazing I have been looking around the web and your first 3 lines gave me my light bulb moment. It makes much more sense to me now. — Michael, Jan 10 '17 at 05:15
Welcome, for a web scenario, they are like an Ajax call, with some subtle differences, which are .Net specific — Mrinal Kamboj, Jan 10 '17 at 05:16
@MrinalKamboj in the above case if we need to display some busyindicator parallely how we can do that? in my case case busyindicator is getting stuck or showing flickering effect. does anyone have solution or any hint ? — Suchith, Aug 19 '19 at 15:11
@Suchith shall be feasible just that Ui control like busy indicator can only be executed on Ui thread, not a thread pool thread, check this [link](https://www.codeproject.com/Articles/567356/Async-Await-in-WPF-to-invoke-WCF-with-Busy-Indicat) — Mrinal Kamboj, Aug 20 '19 at 12:42

Alex Wiese · Answer 2 · 2017-01-10T08:27:18.553

7

If you want them to run in parallel you could spawn a new Task for each operation and then await the completion of all using Task.WhenAll.

private async Task<List<TrelloCard>> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
{
    List<Task<TrelloCard>> tasks = new List<Task<TrelloCard>>();

    foreach (var job in jobs)
    {
        tasks.Add(ProcessCards(job));
    }

    var results = await Task.WhenAll(tasks);

    return results.ToList();
}


private Task<TrelloCard> ProcessCards(Job job)
{
    return Task.Run(() =>
    {
        System.Threading.Thread.Sleep(2000);  //Just for examples sake

        return new TrelloCard();
    });
}

edited Jan 10 '17 at 08:27

answered Jan 10 '17 at 05:17

Alex Wiese

8,142
6
42
71

Only thing you may want to avoid here is `jobs.ToList()`, as it would cease to be a remote operation and all the data will be loaded in the process memory, thus a compute bound operation is more appropriate than IO bound operation. It would actually defeat the purpose of Async-Await. – Mrinal Kamboj Jan 10 '17 at 05:19
@MrinalKamboj yes the only reason to write this code (compute-bound) with async/await would be if it was WPF/WinForms and would be blocking the UI. In an ASP.NET application compute intensive work should not use async/await. – Alex Wiese Jan 10 '17 at 05:25
that's not the case even in WPF you can start a call on `Threadpool` threads, you don't need `Async-Await`, until and unless you are planning to update a `Ui control`, which always need to be done in the `Ui thread context`, else an exception (its easy to achieve using Async-Await, as there's no thread). Ideally no application logic, which does compute intensive work shall use the Async-Await, which is designed only for IO calls. – Mrinal Kamboj Jan 10 '17 at 05:34
@MrinalKamboj no your statement is wrong, async/await is not just for IO calls. It is also designed to prevent blocking in desktop/mobile applications. In WPF async/await is perfect for compute intensive work. You can combine `Task.Run` to use a thread from the `ThreadPool` and use async/await to marshal the continuation onto the UI thread. Without async/await you would have to directly use either the `Dispatcher`, or `TaskScheduler.FromCurrentSynchronizationContext`. Stephen Cleary has a great answer with links [here](http://stackoverflow.com/a/18015586/1411687) that you can read. – Alex Wiese Jan 10 '17 at 06:23
There's no point debating the working of a technology concept, essence can be captured by following article http://blog.stephencleary.com/2013/11/there-is-no-thread.html by Stephen cleary (there's no thread in async operation), that's the reason you are keen to use it in WPF, it frees the Ui thread and since no other thread is involved, so context gets updated in the same context. Ideally its for only for a IO bound operation that you don't want to freeze the Ui. while awaiting, else its always worth waiting for a compute bound operation, since the results are getting processed. – Mrinal Kamboj Jan 10 '17 at 07:13
Another very good link which suggests, that by itself async-await is nothing but taking things in background, implementation scope are many, but ideal one use the IO completion ports, which dispatch a remote call http://stackoverflow.com/a/18443222/1559611 – Mrinal Kamboj Jan 10 '17 at 07:16
@MrinalKamboj "It's always worth waiting for a compute bound operation". Yes exactly, you just made my point - hence the `await` keyword. – Alex Wiese Jan 10 '17 at 08:11
When I say worth waiting it means blocking the Ui thread, which doesn't happen in `Async-Await` – Mrinal Kamboj Jan 10 '17 at 08:17
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/132763/discussion-between-alexw-and-mrinal-kamboj). – Alex Wiese Jan 10 '17 at 08:33

score 1 · Answer 3 · answered Jan 10 '17 at 06:05

jobs.ToList() is just wasting memory. It's already IEnumerable so can be used in a foreach.

ProcessCards doesn't compile. You need something like this

    private Task<TrelloCard> ProcessCards(Job job)
    {
        return Task.Run(() =>
        {
            System.Threading.Thread.Sleep(2000);  //Just for examples sake

            return new TrelloCard();
        });
    }

Now you want ProcessJobs to

create a ProcessCards task for each job
wait for all tasks to finish

return a sequence of TrelloCard

private async Task<List<TrelloCard>> ProcessJobs(IQueryable<IGrouping<CardGrouping, Job>> jobs)
{
    return await Task.WhenAll(jobs.Select(ProcessCards));
}

+1, As I have pointed out in one of the comments earlier, for `Async-Await` usage its important not to do `jobs.ToList()`, since once its bought in the local memory, then it can be a simple Parallel processing instead of Async processing — Mrinal Kamboj, Jan 10 '17 at 07:23

Running Async Foreach Loop C# async await

3 Answers3

Linked