10

I read the how to by Microsoft at How to: Make Multiple Web Requests in Parallel by Using async and await (C#) and found:

private async Task CreateMultipleTasksAsync()  
{  
    // Declare an HttpClient object, and increase the buffer size. The  
    // default buffer size is 65,536.  
    HttpClient client =  
        new HttpClient() { MaxResponseContentBufferSize = 1000000 };  

    // Create and start the tasks. As each task finishes, DisplayResults   
    // displays its length.  
    Task<int> download1 =   
        ProcessURLAsync("http://msdn.microsoft.com", client);  
    Task<int> download2 =   
        ProcessURLAsync("http://msdn.microsoft.com/library/hh156528(VS.110).aspx", client);  
    Task<int> download3 =   
        ProcessURLAsync("http://msdn.microsoft.com/library/67w7t67f.aspx", client);  

    // Await each task.  
    int length1 = await download1;  
    int length2 = await download2;  
    int length3 = await download3;  

    int total = length1 + length2 + length3;  

    // Display the total count for the downloaded websites.  
    resultsTextBox.Text +=  
        string.Format("\r\n\r\nTotal bytes returned:  {0}\r\n", total);  
}

I understand this code, but my question is: How do I modify this to scale it to like, let's say a hundred or a thousand?

Buyo
  • 173
  • 1
  • 1
  • 10
  • 1
    With all due respect, even if a **word** *parallel* might have appeared elsewhere, the nature of the process execution is a "just"-`[CONCURRENT]` process-flow behaviour inside a realm of a distributed-system environment, definitely **not** a true `[PARALLEL]` process scheduling. Hope this may help you refine your further C/S efforts >>> https://stackoverflow.com/tags/parallel-processing/info – user3666197 Jan 02 '18 at 09:51
  • Use-cases, like **having about a 100- or 1000-lane** *parallel* **Highway built**, could be for sure declared using some *smart* syntax-constructors, if **( then having all cars have to wait at port for a ferry to first arrive ) damages all such efforts & all overhead costs got paid in vain** making in fact a **negative** net effect of such design. ( On *"negative"* effects of overheads paid on the expected speedups **see:** >>> https://stackoverflow.com/revisions/18374629/3 ) – user3666197 Jan 02 '18 at 09:59

2 Answers2

12

I can't comment due to the restriction on my account, but in reply to @CreativeManix's answer,

List<Task> tasks = new List<Task>();

Task.WaitAll(tasks) will not accept a List of Tasks. One of its override accepts an Array of Tasks.

Task.WaitAll(tasks.ToArray())
batuzai04123
  • 379
  • 5
  • 12
8

You can invoke async calls in loop. Each call could return a Task, and you have to wait for all Tasks to complete

var requestInfoCollection = new RequestInfo[]
{
     new RequestInfo("http://url1","GET"),
     new RequestInfo("http://url2","GET"),
     new RequestInfo("http://url2","POST")
};
List<Task> tasks = new List<Task>();
foreach(var requestInfo in requestInfoCollection)
{
   tasks.Add(ProcessURLAsync(requestInfo))
}
Task.WaitAll(tasks);

The above will invoke multiple requests and waits for results, however async\await is helpful to release thread to application to use while performing external invocations (http, db etc...). But scaling is depends on your hardware, and your application architecture.

CreativeManix
  • 2,162
  • 1
  • 17
  • 29