0

I have the following method inside my asp.net mvc-5 web application :-

public async Task <List<Details>> Get()
{
    Parallel.ForEach(resourcesinfo.operation.Details,new ParallelOptions { MaxDegreeOfParallelism = 20 }, (c) =>
    {
        ResourceAccountListInfo resourceAccountListInfo = new ResourceAccountListInfo();
        using (WebClient wc = new WebClient()) 
        {
            string url = currentURL + "resources/" + c.RESOURCEID + "/accounts?AUTHTOKEN=" + pmtoken;
            string tempurl = url.Trim();

            var json =  await wc.DownloadStringTaskAsync(tempurl);
            resourceAccountListInfo = JsonConvert.DeserializeObject<ResourceAccountListInfo>(json);
        }
        //code goes here

but I am getting this error :-

The 'await' operator can only be used within an async lambda expression. Consider marking this lambda expression with the 'async' modifier

so can anyone adivce on this please ?

René Vogt
  • 43,056
  • 14
  • 77
  • 99
John John
  • 1
  • 72
  • 238
  • 501
  • @Spivonious i would be careful with that duplicate, you can't use a `async` delegate with `Parallel.ForEach` so the duplicate gives bad advice for this case. – Scott Chamberlain Jun 28 '16 at 13:41
  • `Parallel.Foreach` is not designed to parallelize asynchronous operations; it's designed to parallelize synchronous operations. You shouldn't be using it at all. – Servy Jun 28 '16 at 13:41
  • 1
    Aaah, as @scottchamberlain mentioned in my now deleted answer, you can't. `Parallel.ForEach` doesn't have an overload that accepts a Task. I guess the main problem here is that you're mixing up patterns for async operations from two different paradigms (Tasks and Pfx). If you're handling `async` stuff, then better to go full on Tasks. Here's some C# like pseudocode which illustrates: `Task.WaitAll(resourcesinfo.operation.Details.Select(x => CallAsyncMethod(x)).ToArray())` –  Jun 28 '16 at 13:43
  • Related? http://stackoverflow.com/questions/23137393/parallel-foreach-and-async-await?rq=1 – Spivonious Jun 28 '16 at 13:44
  • @will thanks for your reply.. so now if i want to run async methods (DownloadStringTaskAsync in my case) in a parallel way ,, what are the approaches i can follow ? – John John Jun 28 '16 at 14:11
  • 1
    There are a number of patterns that work. Here's a starting point. http://weblogs.asp.net/cibrax/await-whenall-waitall-oh-my The TPL library also sounds like a great way to go. –  Jun 28 '16 at 14:13
  • @Will or i can take the short route and use sync methods instead of async methods inside the Parallel.Foreach is this correct ? the problem is that i find the whole TPL syntax a bit confusing to me ,, maybe because i have never use it before .. – John John Jun 28 '16 at 16:25

1 Answers1

3

Parallel.ForEeach is not designed to work with asynchronous functions, you need to use more modern classes like the ones in TPL Dataflow. You get it by installing the NuGet package to your project Microsoft.Tpl.Dataflow. You could recreate your previous code as

private const int MAX_PARALLELISM = 20

public async Task <List<Details>> Get()
{
    var block = new ActionBlock<Entry>(async (c) => 
        {
            ResourceAccountListInfo resourceAccountListInfo = new ResourceAccountListInfo();
            using (WebClient wc = new WebClient()) 
            {
                string url = currentURL + "resources/" + c.RESOURCEID + "/accounts?AUTHTOKEN=" + pmtoken;
                string tempurl = url.Trim();

                var json =  await wc.DownloadStringTaskAsync(tempurl);
                resourceAccountListInfo = JsonConvert.DeserializeObject<ResourceAccountListInfo>(json);
            }
            //code goes here
        }
        ,new ExecutionDataflowBlockOptions 
                   { 
                         MaxDegreeOfParallelism = MAX_PARALLELISM
                   });

    foreach(var entry in resourcesinfo.operation.Details)
    {
        await block.SendAsync(entry);
    }

    block.Complete();
    await block.Completion;

    //More code here    
}

After thinking for a bit, here is a slightly more complicated version that does the entire pipeline from reading the records in to returning a result from Get()

private const int MAX_PARALLELISM = 20

public async Task<List<Details>> Get()
{
    List<Details> result = new List<Details>();

    var getRecordBlock = new TransformBlock<Entry, ResourceAccountListInfo>(async (c) =>
        {
            ResourceAccountListInfo resourceAccountListInfo = new ResourceAccountListInfo();
            using (WebClient wc = new WebClient())
            {
                string url = currentURL + "resources/" + c.RESOURCEID + "/accounts?AUTHTOKEN=" + pmtoken;
                string tempurl = url.Trim();

                var json = await wc.DownloadStringTaskAsync(tempurl);
                resourceAccountListInfo = JsonConvert.DeserializeObject<ResourceAccountListInfo>(json);
            }
            return resourceAccountListInfo;
        }
        , new ExecutionDataflowBlockOptions
        {
            MaxDegreeOfParallelism = MAX_PARALLELISM
        });

    //Defaults to MaxDegreeOfParallelism = 1
    var addToListBlock = new ActionBlock<ResourceAccountListInfo>(info =>
    {
        Details detail = TurnResourceAccountListInfoInToDetails(info);
        result.Add(detail);
    });

    getRecordBlock.LinkTo(addToListBlock, new DataflowLinkOptions { PropagateCompletion = true});

    foreach (var entry in resourcesinfo.operation.Details)
    {
        await getRecordBlock.SendAsync(entry);
    }

    getRecordBlock.Complete();
    await addToListBlock.Completion;

    return result;
}
Community
  • 1
  • 1
Scott Chamberlain
  • 124,994
  • 33
  • 282
  • 431
  • 1
    To anyone reading this, do you know of a better term than "asynchronously block"? I don't like using that phrase in the answer. – Scott Chamberlain Jun 28 '16 at 13:55
  • i did not use TPL Dataflow before,, so is there any out of the box appraoch to have async methods inside parallel.foreach ? for example if i use sync method (DownloadString) can i use it inside a parallel.foreach ? i am trying to find a more out of the box approach? i mean what i can do other than installing TPL Dataflow ?? – John John Jun 28 '16 at 14:03
  • 1
    TPL Dataflow is part of .Net. It just does not ship with the .net installer. Microsoft is moving to that model for .net, it is not "3rd party" and you are not "installing" anything, just include the NuGet package and you are done. – Scott Chamberlain Jun 28 '16 at 14:05
  • so let say i use .Result instead of await/async will this cause any problem ? i mean to have Parallel.foreach and inside it to have wc.DownloadStringTaskAsync(tempurl).Result ??? – John John Jun 28 '16 at 14:08
  • 1
    If you are just going to use synchronous methods just do `wc.DownloadString(tempurl)` don't get async involved at all. However to get the best performance you really should just add the NuGet package to your solution. – Scott Chamberlain Jun 28 '16 at 14:12
  • not sure what is the problem of having async methods inside parallel.foreach ,, why i should not mix both of them ? – John John Jun 28 '16 at 14:14
  • Create a new winforms project and put a button on it. Add [this codebehind](https://gist.github.com/leftler/eae51fdf68d5a2044b04edc71479339e) and wire up the button. when you press it you will deadlock the program because you called `.Result` on a thread that had a `SynchronizationContext`. The class `Parallel.ForEach` will use the calling thread as one of the threads to do it's work, if that thread is a UI thread it will propagate the context, if that happens you can deadlock your code by using `.Result` – Scott Chamberlain Jun 28 '16 at 14:21
  • 1
    @RenéVogt I don't know if you are still watching this answer or not but you may want to check out my previous comment, it shows why doing `.Result` in a `Parallel.Foreach` can be dangerous. – Scott Chamberlain Jun 28 '16 at 14:28
  • that sound valid.. so i will expole the TPL DF , but now inside my code after I get the resourceAccountListInfo from deserilizing the return json inside each foreach iteration , i am getting some values from it as follow var username = resourceAccountListInfo.operation.Details.CUSTOMFIELD.Where(a=>a.Type="UserName").First().Value; so not sure where i need to place the other code which are inside the foreach ? so in this case i will get a username value inside each of the foreach iteration ?? – John John Jun 28 '16 at 14:34
  • It would go inside `TurnResourceAccountListInfoInToDetails` in my 2nd example, That function turns a `ResourceAccountListInfo` record in to a `Details` record – Scott Chamberlain Jun 28 '16 at 14:49
  • so why i need to define this foreach (var entry in resourcesinfo.operation.Details) { await getRecordBlock.SendAsync(entry); } what is the purpose exactly ? – John John Jun 28 '16 at 15:01
  • 1
    That is what pushes data in to `getRecordBlock`, it takes the place of the first parameter of your old `Parallel.ForEach` – Scott Chamberlain Jun 28 '16 at 15:02
  • now inside my original code i have a comment "//code goes here" so where exactly this will go inside your code ? i am a bit confused on where the foreach iteration buisness logic should go ? and is the getRecordBlock.Complete(); & await addToListBlock.Completion; performed at the iteration level or it will apply to the whole foreach for one time only ?? – John John Jun 28 '16 at 15:12
  • 1
    `//Code goes here` goes inside of `TurnResourceAccountListInfoInToDetails`, the `getRecordBlock.Complete()` signals "I am done adding items to `getRecordBlock`, the `await addToListBlock.Completion` signals that `addToListBlock` has finished processing the last item that was in the queue. Once you get past that `await` you know `result` has been filled and is ready to be returned to the user. – Scott Chamberlain Jun 28 '16 at 15:20
  • ok i see... so let me summarize your points. First Point) now i can use the parallel.foreach but with sync method (wc.DownloadString) this should not cause any problem?,, Second Point ) if i want to use async methods (wc.DownloadStringTaskAsync) then i should NOT use it inside a Parallel.Foreach instead i should use modern classes like the ones in TPL Dataflow,, .. is this correct ? – John John Jun 28 '16 at 16:24