1

If I have to make simultaneously 100 of async queries to some URL. My code looks like this:

var tasks = requests.Select(x => httpClient.GetStringAsync(x)).ToList();
return Task.WhenAll(tasks);

Should I care about the limitation of concurrent calls? It means, don't create 100 tasks simultaneously, but do it by chunks. Good post on this subject is here.

I prefer rely on ServicePointManager.DefaultConnectionLimit instead. Windows OS limits the number of outbound connections. Look here. So, I'm creating 100 Tasks and only 10 of them (DefaultConnectionLimit default) are running in parallel.

Inspired by How to limit the amount of concurrent async I/O operations?.

Stefan
  • 17,448
  • 11
  • 60
  • 79
dibr77
  • 29
  • 3
  • I'm not sure I understand the question. Could you try to reword it? Are you trying to get it working without changing the value of `ServicePointManager.DefaultConnectionLimit`? – 41686d6564 stands w. Palestine Jul 13 '19 at 18:04
  • My point is: I don't need to think about how to limit amount of Tasks (requests) running in parallel. SerivePointManager do it for me. I create 100 Tasks, but only 10 of them are running in parallel. The others are waiting. I don't change the value for DefaultConnectionLimit, because I don't know which value gives me the best performance. The default for ASP.NET is 10 based on the following article: https://learn.microsoft.com/en-us/dotnet/api/system.net.servicepointmanager.defaultconnectionlimit?view=netframework-4.8 – dibr77 Jul 14 '19 at 05:00
  • yes you do need to think about it because if you just create 100 tasks and your requests are long running then you'll get a lot of request timeout exceptions for all of those requests that were sitting in a ServicePointManagers queue without even getting a change to get their live connection. That's why you should do your own batching logic and not create all the tasks at once. ServicePointManager is not your batches manager. It just tells how many outbound connections you can have live concurrently. If you create more tasks, they go to queue with no guarantee they won't timeout before. – dee zg Jul 14 '19 at 19:48

1 Answers1

3

Yes, you should care about the limitation of concurrent calls; but only as per use case.

Bottom line: it depends on your infrastructure.

For example, if you are on a shared network connection; fetching a lot of data from different sources can clog your network.

Another example: writing to a network server might be more efficient when doing so in larger chuncsk, with fewer simultaneous connections.

Depending on where you run this, it worth to know how many possible parallel tasks are being executed.

If you know that, you can tweak it to be reasonable within the boundaries of your environment. This depends on the hardware and the possible load....

That's basically what's meant in the article by:

The optimum number of connections depends on the actual conditions in which the application runs. Increasing the number of connections available to the application may not affect application performance. To determine the impact of more connections, run performance tests while varying the number of connections.

The default of the HttpWebRequest, as described, is pretty low. Which would result in a pretty good "general approach" scenario, even on home/consumers internet connections. If your situation is e.g.: high performant server to server communication, then, different values would make more sense.

The key is to know the environment/context and it's capabilities.

Then you can determine a good number of parralel operations.

And that's why you must care about it, and sometimes limit it :-)

Stefan
  • 17,448
  • 11
  • 60
  • 79
  • 2
    Hi Stefan, I'm not sure if we are on the same page, but you gave me an idea about the problem with my code. With my 100 queries I took all the outbound connections and may be for a long time. No other requests can be processed till this 100 is finished. For my server it can be a problem. Many thanks. – dibr77 Jul 14 '19 at 05:17