16

I am attempting to run async methods from a synchronous method. But I can't await the async method since I am in a synchronous method. I must not be understanding TPL as this is the fist time I'm using it.

private void GetAllData()
{
    GetData1()
    GetData2()
    GetData3()
}

Each method needs the previous method to finish as the data from the first is used for the second.

However, inside each method I want to start multiple Task operations in order to speed up the performance. Then I want to wait for all of them to finish.

GetData1 looks like this

    internal static void GetData1 ()
    {
        const int CONCURRENCY_LEVEL = 15; 
        List<Task<Data>> dataTasks = new List<Task<Data>>();
        for (int item = 0; item < TotalItems; item++)
        {
            dataTasks.Add(MyAyncMethod(State[item]));
        }
        int taskIndex = 0;
        //Schedule tasks to concurency level (or all)
        List<Task<Data>> runningTasks = new List<Task<Data>>();
        while (taskIndex < CONCURRENCY_LEVEL && taskIndex < dataTasks.Count)
        {
            runningTasks.Add(dataTasks[taskIndex]);
            taskIndex++;
        }

        //Start tasks and wait for them to finish
        while (runningTasks.Count > 0)
        {
            Task<Data> dataTask = await Task.WhenAny(runningTasks);
            runningTasks.Remove(dataTask);
            myData = await dataTask;


            //Schedule next concurrent task
            if (taskIndex < dataTasks.Count)
            {
                runningTasks.Add(dataTasks[taskIndex]);
                taskIndex++;
            }
        }
        Task.WaitAll(dataTasks.ToArray()); //This probably isn't necessary
    }

I am using await here but get an Error

The 'await' operator can only be used within an async method. Consider marking this method with the 'async' modifier and changing its return type to 'Task'

However, if I use the async modifier this will be an asynchronous operation. Therefore, if my call to GetData1 doesn't use the await operator won't control go to GetData2 on the first await, which is what I am trying to avoid? Is it possible to keep GetData1 as a synchronous method that calls an asynchronous method? Am I designing the Asynchronous method incorrectly? As you can see I'm quite confused.

This could be a duplicate of How to call asynchronous method from synchronous method in C#? However, I'm not sure how to apply the solutions provided there as I'm starting multiple tasks, want to WaitAny, do a little more processing for that task, then wait for all tasks to finish before handing control back to the caller.

UPDATE

Here is the solution I went with based on the answers below:

    private static List<T> RetrievePageTaskScheduler<T>(
        List<T> items,
        List<WebPageState> state,
        Func<WebPageState, Task<List<T>>> func)
    {
        int taskIndex = 0;

        // Schedule tasks to concurency level (or all)
        List<Task<List<T>>> runningTasks = new List<Task<List<T>>>();
        while (taskIndex < CONCURRENCY_LEVEL_PER_PROCESSOR * Environment.ProcessorCount
            && taskIndex < state.Count)
        {
            runningTasks.Add(func(state[taskIndex]));
            taskIndex++;
        }

        // Start tasks and wait for them to finish
        while (runningTasks.Count > 0)
        {
            Task<List<T>> task = Task.WhenAny(runningTasks).Result;
            runningTasks.Remove(task);

            try
            {
                items.AddRange(task.Result);
            }
            catch (AggregateException ex)
            {
                /* Throwing this exception means that if one task fails 
                 * don't process any more of them */

                // https://stackoverflow.com/questions/8853693/pattern-for-implementing-sync-methods-in-terms-of-non-parallel-task-translating
                System.Runtime.ExceptionServices.ExceptionDispatchInfo.Capture(
                    ex.Flatten().InnerExceptions.First()).Throw();
            }

            // Schedule next concurrent task
            if (taskIndex < state.Count)
            {
                runningTasks.Add(func(state[taskIndex]));
                taskIndex++;
            }
        }

        return items;
    }
Community
  • 1
  • 1
Harrison
  • 3,843
  • 7
  • 22
  • 49
  • The `Task` type has two completely different use cases: asynchronous work (e.g., `async`/`await`) and parallel processing (e.g., `Task.Factory.StartNew`/`Task.WaitAll`). What kind of work are you doing (i.e., is it CPU-bound)? – Stephen Cleary Sep 25 '13 at 16:34
  • I am retreiving a web page (I/O bound) and then processing it CPU bound. The MyAsyncMethod is constructed using two awaits one for the IO bound work and one for the CPU bound work. – Harrison Sep 25 '13 at 16:40

3 Answers3

9

Task<TResult>.Result (or Task.Wait() when there's no result) is similar to await, but is a synchronous operation. You should change GetData1() to use this. Here's the portion to change:

Task<Data> dataTask = Task.WhenAny(runningTasks).Result;
runningTasks.Remove(dataTask);
myData = gameTask.Result;
Tim S.
  • 55,448
  • 7
  • 96
  • 122
  • Thanks for helping a newbie. This looks like a great solutions and certainly fixes my compile error. I assume you removed the `Task.WaitAll(dataTasks.ToArray());` as you agreed it wasn't necessary since when all of the WhenAny are completed this is guaranteed to already have happened. – Harrison Sep 25 '13 at 16:51
  • @Harrison actually, I was just highlighting the part where you had `await`, but now that you mention it, I think you're right about that. – Tim S. Sep 25 '13 at 16:53
  • if I am return the result from the function, Can I directly do `return Task.WhenAny(runningTasks).Result.Result` ? – Anuj Pandey Jan 31 '14 at 15:16
  • If experiencing UI deadlock: http://stackoverflow.com/questions/14485115/synchronously-waiting-for-an-async-operation-and-why-does-wait-freeze-the-pro – Jeremy Ray Brown Oct 20 '16 at 19:55
9

First, I recommend that your "internal" tasks not use Task.Run in their implementation. You should use an async method that does the CPU-bound portion synchronously.

Once your MyAsyncMethod is an async method that does some CPU-bound processing, then you can wrap it in a Task and use parallel processing as such:

internal static void GetData1()
{
    // Start the tasks
    var dataTasks = Enumerable.Range(0, TotalItems)
        .Select(item => Task.Run(() => MyAyncMethod(State[item]))).ToList();

    // Wait for them all to complete
    Task.WaitAll(dataTasks);
}

Your concurrency limiting in your original code won't work at all, so I removed it for simpilicity. If you want to apply a limit, you can either use SemaphoreSlim or TPL Dataflow.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • 1
    You're closing over the loop variable when you likely didn't mean to. – Servy Sep 25 '13 at 16:52
  • Thanks for the help. In reading Implementing the Task-based Asynchronous Pattern [http://msdn.microsoft.com/en-us/library/hh873177.aspx] it says "In the .NET Framework 4.5, use the static Task.Run method as a shortcut to TaskFactory.StartNew" for CPU bound work. It sounds like you are recommending a different approach. Am I understanding you correctly? Since my method is a hyrbid of CPU and I/O are you recommending me to split it in two? Also, can you explain why the concurrency limiting won't work? Thanks again. – Harrison Sep 25 '13 at 17:06
  • 1
    You stated that `MyAsyncMethod` has a CPU-bound portion, so you need to use `Task.Run`. However, you should use it when *calling* the method, not in its *implementation*. You don't need to split it in two, but you do have the (unusual) case of an `async` method that is also CPU-bound, so you should document it well. The concurrency limiting won't work because as soon as you call `MyAsyncMethod`, the method is already running. – Stephen Cleary Sep 25 '13 at 17:30
  • @StephenCleary - Why is there a need for Task.Run in the Select? Why could it not just be `.Select(item => MyAsyncMethod(State[item]))`? – Seth Flowers Apr 03 '18 at 19:42
  • @sethflowers: The op indicated that `MyAsyncMethod` has a nontrivial CPU-bound portion and thus wanted it to run on the thread pool. This is the [complex use case of `Task.Run`](https://blog.stephencleary.com/2013/11/taskrun-etiquette-examples-even-in.html). – Stephen Cleary Apr 04 '18 at 15:01
  • @StephenCleary - thanks - your blog post helped me to better understand the problem the OP was having. – Seth Flowers Apr 04 '18 at 16:05
0

You can call the following:

GetData1().Wait();
GetData2().Wait();
GetData3().Wait();
Adam Modlin
  • 2,994
  • 2
  • 22
  • 39
  • 7
    Note that if there is a current `SynchronizationContext` defined this code will deadlock. – Servy Sep 25 '13 at 16:46