Effects of async within a parallel for loop

Question

I am going to start by saying that I am learning about mulithreading at the moment so it may be the case that not all I say is correct - please feel free to correct me as required. I do have a reasonable understanding of async and await.

My basic aim is as follows: I have a body of code that currently takes about 3 seconds. I am trying to load some data at the start of the method that will be used right at the end. My plan is to load the data on a different thread right at the start - allowing the rest of the code to execute independently. Then, at the point that I need the data, the code will wait if the data is not loaded. So far this is all seems to be working fine and as I describe.

My question relates to what happens when I call a method that is async, within a parallel for loop, without awaiting it.

My code follows this structure:

public void MainCaller()
    {
        List<int> listFromThread = null;
        var secondThread = Task.Factory.StartNew(() =>
        {
            listFromThread = GetAllLists().Result;
        });

        //Do some other stuff

        secondThread.Wait();
        //Do not pass this point until this thread has completed
    }

    public Task<List<int>> GetAllLists()
    {
        var intList = new List<int>(){ /*Whatever... */};
        var returnList = new List<int>();
        Parallel.ForEach(intList, intEntry =>
        {
            var res = MyMethod().Result;
            returnList.AddRange(res);
        });

        return Task.FromResult(returnList);
    }

    private async Task<List<int>> MyMethod()
    {
        var myList = await obtainList.ToListAsync();
    }

Note the Parallel for Loop calls the async method, but does not await it as it is not async itself.

This is a method that is used elsewhere, so it is valid that it is async. I know one option is to make a copy of this method that is not async, but I am trying to understand what will happen here.

My question is, can I be sure that when I reach secondThread.Wait(); the async part of the execution will be complete. Eg will wait to know wait for the async part to complete, or will async mess up the wait, or will it work seamlessly together?

It seems to me it could be possible that as the call to MyMethod is not awaited, but there is an await within MyMethod, the parallel for loop could continue execution before the awaited call has completed?

Then I think, as it is assigning it by reference, then once the assigning takes place, the value will be the correct result.

This leads me to think that as long as the wait will know to wait for the async to complete, then there is no problem - hence my question.

I guess this relates to my lack of understanding of Tasks?

I hope this is clear?

My opinion, is that the whole arch is messed up. First, eliminate `.Wait`, then `.Result`s and then `Parallel.ForEach`... I dont feel like they are needed here. Now, you should really learn a difference between async and parallel. — AgentFire, Jul 31 '19 at 09:45
@AgentFire can you explain why I should remove all of these things? — Alex, Jul 31 '19 at 09:46
In general you should not use `.Result` and/or `.Wait` on `Task`s because this results in a blocking call which destroys the whole purpose of async/await. Whenever you're using `.Wait` or `.Result` think about how you could do it differently. Also returning some `Task` if you only create it via `Task.FromResult` and don't use any `await`s should be avoided since it only gives you worse performance and no benefits over just returning it normally without wrapping it into a task. — Joelius, Jul 31 '19 at 09:46
@Joelius I am not trying to leverage the benefits of await - I am just trying to understand the points I raise in my question — Alex, Jul 31 '19 at 09:49
Actually, in your parallel.for loop, you are calling MyMethod synchronously since you are calling 'MyMethod().Result. So each task that is started from this loop will block untill MyMethod is finished. — Johan Donne, Jul 31 '19 at 09:50
@JohanDonne that is what I am trying to establish - can you elaborate more - maybe in answer? — Alex, Jul 31 '19 at 09:51
The important point is if you use Parallel and async correctly you wouldn't need to ask these questions. Also you don't have any async calls in your parallel loop because you call `.Result` and it seems like you think this is still async when it is in fact fully blocking. It's difficult topic this is why I would advise you to first learn why and how to remove blocking calls on async methods and then ask any remaining questions with the new code (without .Wait etc). — Joelius, Jul 31 '19 at 09:54

score 1 · Accepted Answer · answered Jul 31 '19 at 10:07

In your code there is no part that is executed asynchrounously.

In MainCaller, you start a Task and immediately Wait for it to finished. This is a blocking operation which only introduces the extra overhead of calling GetAllLists in another Task.
In this Task you call You start a new Task (by calling GettAllLists) but immediately wait for this Task to finish by waiting for its Result (which is also blocking).
In the Task started by GetAllLists you have the Parallel.Foreach loop which starts several new Tasks. Each of these 'for' Tasks will start another Task by calling MyMethod and immediately waiting for its result.

The net result is that your code completely executes synchronously. The only parallelism is introduced in the Parallel.For loop.

Hint: a usefull thread concerning this topic: Using async/await for multiple tasks

Additionally your code contains a serious bug: Each Task created by the Parallel.For loop will eventually add its partial List to the ReturnList by calling AddRange. 'AddRange' is not thread safe, so you need to have some synchronisation mechanism (e.g. 'Lock') or there is the possibility that your ReturnList gets corrupted or does not contain all the results. See also: Is the List<T>.AddRange() thread safe?

Effects of async within a parallel for loop

1 Answers1