1

I am stuck in a situation where I have to download multiple files from a list of urls. I found Parallel.ForEach as a solution and it really works faster than a simple foreach. But this solution is not waiting till all downloads are done.

List<string> list = new List<string>() {"url1","url2","url3",... };

public async void downloadSimpleFile(string fileUrl) {
    //download file logic
}

await Task.Factory.StartNew(() => Parallel.ForEach(list,
    new ParallelOptions { MaxDegreeOfParallelism = 10 }, downloadSimpleFile));

I was also tying:

public async Task<bool> downloadSimpleFile(string fileUrl) {
    //download file logic
    return true;
}

Now how can I use downloadSimpleFile in Parallel.ForEach, because up mentioned code is not working anymore?

I was reading some posts about async-await in Parallel like this or this but I'm not getting the idea. Any ideas are apreciated.

Community
  • 1
  • 1
Vasile Doe
  • 1,674
  • 1
  • 24
  • 40
  • 1
    It works faster *because* it's not waiting. As it says in the questions you link to, you probably don't want to do this. Just create all the tasks and use `Task.WhenAll` to await completion. – Charles Mager Jun 01 '16 at 11:19

1 Answers1

3

Use the Task.WhenAll method. Following code should work:

public async Task<bool> downloadSimpleFile(string fileUrl)
{
    //download file logic
    return true;
}

public async Task ProcessList()
{
    var list = new List<string> {"url1", "url2", "url3" };
    var downloadFiles = list.Select(downloadSimpleFile).ToArray();

    var result = await Task.WhenAll(downloadFiles);
}
Abbas
  • 14,186
  • 6
  • 41
  • 72
  • I tested this method and looks like tasks are started not in parallel at the same time, there is a difference about 0.2 sec in each `downloadSimpleFile` call – Vasile Doe Jun 02 '16 at 11:19