3

There is an existing third party Rest API available which would accept one set of input and return the output for the same. (Think of it as Bing's Geo coding service, which would accept address and return location detail)

My need would be is to call this API multiple times (say 500-1000) for a single asp.net request and each call may take close to 500ms to return.

I could think of three approaches on how to do this action. Need your input on which could be best possible approach keeping speed as criteria.

1. Using Http Request in a for loop

Write a simple for loop and for each input call the REST API and add the output to the result. This by far could be the slowest. But there is no overhead of threads or context switching.

2. Using async and await

Use async and await mechanisms to call REST Api. It could be efficient as thread continues to do other activites while waiting for REST call to return. The problem I am facing is that, as per recommendations I should be using await all the way to the top most caller, which is not possible in my case. Not following it may lead to dead locks in asp.net as mentioned here http://msdn.microsoft.com/en-us/magazine/jj991977.aspx

3. Using Task Parallel Library

Using a Parallel.ForEach and use the Synchronuos API to invoke the Server parallely and use ConcurrentDictionary to hold the result. But may result in thread overhead

Also, let me know is there any other better way to handle things. I understand people might suggest to track performance for each approach, but would like to understand how people has solved this problem before

Ramesh
  • 13,043
  • 3
  • 52
  • 88
  • 1
    Instead of collecting result in Http Request and sending back to client, why don't you let client make multiple requests and forward them to server, let the client manage asynchronous logic as JavaScript does it pretty well. – Akash Kava Feb 27 '14 at 09:44
  • @AkashKava The client finalizes the entries by clicking a button. I need to invoke the service api and store the results in DB for each entry. Asking JS to do this will not be efficient. – Ramesh Feb 27 '14 at 09:47
  • @AkashKava Also, Concurrent connections to a domin is restricted and varies across browsers and version – Ramesh Feb 27 '14 at 09:54
  • 1
    Why exactly can't you use async all the way? – svick Feb 27 '14 at 17:10
  • @svick - I am just modifying a part of an existing system my caller assumed it will be a sync operation. Hence can not modify all the way to top. – Ramesh Feb 27 '14 at 17:44
  • @svick - I am though currently thinking of modifying all the way till controller to make it async – Ramesh Feb 27 '14 at 17:54
  • @Ramesh, this might be close to what you're looking for: http://stackoverflow.com/a/22002868/1768303 – noseratio Feb 27 '14 at 23:48
  • @Noseratio The question you have linked to explains one request waiting for a long time. In this scenario there are multiple requests coming back in very short span of time. – Ramesh Feb 28 '14 at 04:56
  • @Ramesh, it's simply a matter of doing this: `await Task.WhenAll(requests.Select(r => httpClient.GetStringAsync(r.url)).ToArray())`, if I understood your question correctly. – noseratio Feb 28 '14 at 05:01

2 Answers2

4

The best solution is to use async and await, but in that case you will have to take it async all the way up the call stack to the controller action.

The for loop keeps it all sequential and synchronous, so it would definitely be the slowest solution. Parallel will block multiple threads per request, which will negatively impact your scalability.

Since the operation is I/O-based (calling a REST API), async is the most natural fit and should provide the best overall system performance of these options.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
2

First, I think it's worth considering some issues that you didn't mention in your question:

  • 500-1000 API calls sounds like quite a lot. Isn't there a way to avoid that? Doesn't the API have some kind of bulk query functionality? Or can't you download their database and query it locally? (The more open organizations like Wikimedia or Stack Exchange often support this, the more closed ones like Microsoft or Google usually don't.)

    If those options are not available, then at least consider some kind of caching, if that makes sense for you.

  • The number of concurrent requests to the same server allowed at the same time in ASP.NET is only 10 by default. If you want to make more concurrent requests, you will need to set ServicePointManager.DefaultConnectionLimit.

  • Making this many requests could be considered abuse by the service provider and could lead to blocking of your IP. Make sure the provider is okay with this kind of usage.

Now, to your actual question: I think that the best option is to use async-await, even if you can't use it all the way. You can avoid deadlocks either by using ConfigureAwait(false) at every await (which is the correct solution) or by using something like Task.Run(() => /* your async code here */).Wait() to escape the ASP.NET context (which is the simple solution).

Using something like Parallel.ForEach() is not great, because it unnecessarily wastes ThreadPool threads.

If you go with async, you should probably also consider throttling. A simple way to achieve that is by using SemaphoreSlim.

Community
  • 1
  • 1
svick
  • 236,525
  • 50
  • 385
  • 514
  • Thanks @scivk - There is no bulk query / DB access. Would definitely look at caching the values. The concurrent connections to same server is new to me. I will look more in detail for that. I believe the third party has white listed the IP address of our server. Very informative. Thanks – Ramesh Feb 28 '14 at 04:49
  • Can you please explain the need for throttling in case of async also please? – Ramesh Feb 28 '14 at 04:52
  • @Well, setting `ServicePointManager.DefaultConnectionLimit` affects your whole application. So, if you want to use different number of concurrent connections in different parts of your application, throttling might be the right approach. – svick Feb 28 '14 at 10:50
  • I have modified all the way to controller to be async. But I get the below exception `System.InvalidOperationException: An asynchronous module or handler completed while an asynchronous operation was still pending.` I am basically trying to `foreach(var request in requests) { dictionary.add(request.id, ProcessReqAsync(request)); } foreach(var entry in dictionary) { result.add(entry.Key, await entry.value); } return result;` – Ramesh Mar 05 '14 at 10:24
  • I think that should work, you should probably ask a new question about this. – svick Mar 05 '14 at 11:28