5

Frameworks like Node.js, Tornado, and Twisted let developers create server-push applications that supports large number of concurrent hanging requests (10k+). From what I understand, they all achieve this by not creating threads to service each hanging request.

Can AsyncController be used to service large number of inactive concurrent requests?

If so, are there any reasonably large ASP.NET MVC websites using this approach to create long-poll applications?

David H
  • 2,901
  • 1
  • 26
  • 21
  • Any progress with this? For 10k+ hanging requests, it sounds like you'd need to modify both `maxWorkerThreads` and `maxConcurrentRequestsPerCPU`. – bzlm Aug 03 '11 at 11:19

2 Answers2

6

AsyncController is useful (compared to a normal controller) in the following situation:

You have a long running task and this task is normally achieved by some other tier (web service, database, ...) by using I/O Completion Ports. So the request starts on a worker thread, then you call the BeginXXX method which will open an IOCP (if it supports it, if not it is useless as it will simply draw another worker thread) and the worker thread will be immediately returned to the thread pool. During the execution of the long operation no worker threads are being consumed on the server. Once it completes it signals the IOCP, the async controller draws another thread from the pool to simply terminate the request and return the results to the view. There are a few things to note here: the fact that you used an async controller instead of a normal controller makes absolutely no difference to the client: he will still need to wait the same amount of time for the request to finish, don't get the false impression that an async controller will make your slow operation run faster. Simply the worker threads will be monopolized for less time and so potentially making other requests run faster.

To sum up: for fast running requests async controllers won't bring you any benefit compared to normal controllers. For slow requests they could but it will depend on the nature of the long running operation and whether it is CPU or I/O bound. For CPU bound tasks async controllers are not more useful either. But in all cases you should perform extensive load testing on your application.

Here's a very good article on MSDN which explains async requests in ASP.NET.

And here's a blog post which illustrates how an async controller could be used to implement long-polling.

Darin Dimitrov
  • 1,023,142
  • 271
  • 3,287
  • 2,928
  • Yes, I am aware that AsyncControllers can be used to free up worker threads from blocking on long-running IO-bound operations. Instead of that, I want to know whether or not they can be used to hold large number of inactive connections (not IO-bound nor CPU-bound). Sites like Gmail Chat uses long poll to push events to web browsers instead of programming them to poll events from the server at a fixed time interval. – David H Feb 13 '11 at 09:46
  • @David H, I don't quite understand what you mean by a controller holding a large number of inactive connections. In order to implement push functionality instead of poll you could use [HTML 5 WebSocket API](http://dev.w3.org/html5/websockets/). If you block or sleep in a controller no matter whether it is async or not async it is very bad. – Darin Dimitrov Feb 13 '11 at 09:49
  • @Darin Dimitrov, This is what I mean: http://en.wikipedia.org/wiki/Push_technology#Long_polling Also, HTML 5 WebSocket is not available to all web browsers. This technique is more of hack that everyone is using to achieve similar effects. – David H Feb 13 '11 at 10:51
  • @David H, you may take a look here: http://www.sprklab.com/notes/14-long-polling-comet-in-asp.net/ – Darin Dimitrov Feb 13 '11 at 11:01
  • @David H, and here's a blog post which illustrates how to achieve this with ASP.NET MVC async controllers: http://clay.lenharts.net/blog/2010/10/19/websockets-is-cool-but-what-can-you-do-today/ – Darin Dimitrov Feb 13 '11 at 11:08
  • @Darin In the Clay Lenharts article, async controllers only seem to be used to get around the "ASP.Net thread limits" by tricking ASP.NET into switching a thread in the worker process thread pool for a thread elsewhere - to stop long polls from hogging an ASP.NET thread. I find the lack of citations in the article to be a bit troublesome. Does it really work? – bzlm Feb 13 '11 at 13:48
  • @Darin Dimitrov, thanks for the help so far. Two things so far: 1.) the project described in the sprklab.com link doesn't actually use Asp.Net MVC's AsyncController and 2.) the technique shown in Clay Lenharts' article is not very well tested. I tried it out by porting Node.js's chat server (https://github.com/ry/node_chat) using Lenhart's technique and was unable to get more than 5 concurrent users to use the site at the same time. – David H Feb 14 '11 at 18:40
  • @Darin Dimitrov, here's my attempt at the port: https://github.com/buddydvd/aspmvc-chatserver – David H Feb 14 '11 at 19:03
3

I recently wrote a simple example of a Long Polling Chat Server using MVC 3 Async Controllers based on a great article by Clay Lenhart but I haven't had the chance to really test it out with a bunch of connections.

You can use the example on a AppHarbor deployment I set up based on the source from the BitBucket project.

Also, more information available from my blog post explaining the project.

Jacob
  • 3,629
  • 3
  • 36
  • 44
  • Ah, I did similar test as well. I ported Node.js's chat server to ASP.Net MVC and uploaded to app harbor awhile back: http://chat.apphb.com/ It's got race conditions I have not yet fixed. Though this technique works, the question I'm interested in knowing the answer to is whether or not such setup can handle 10k+ users. https://github.com/buddydvd/aspmvc-chatserver/ – David H Apr 14 '11 at 04:24
  • Cool, chat server does seem like the hip thing to write these days. Too bad C# can't do it in 12 lines or whatever, ;). As for concurrent connections, it seems like it would be throttled by the CLR Thread Pool (well, for my implementation, since I'm using Thread.QueueUserWorkItem). I'm going to do some research on how to load test it; if I find anything useful I'll post it here. – Jacob Apr 14 '11 at 13:48
  • @Jacob, `QueueUserWorkItem` with 10k+ hanging requests? Surely the worker thread pool would be enormous? – bzlm Aug 03 '11 at 11:21
  • @bzlm sure, but something has to wait around for the trigger to happen. Do you have an alternative approach to this problem , especially since you downvoted? I'm open to suggestion. In .Net 4.0 the thread pool is essentially limited by memory size (http://stackoverflow.com/questions/2095805/maximum-queued-elements-in-threadpool-queueuserworkitem/2095846#2095846) but I'm not sure how that translates to quantity and whether 10k would be too much for a standard build server. – Jacob Aug 03 '11 at 18:39
  • @Jacob, if you just increase the number of outstanding operations on the `AsyncManager` and then return, and then decrease the number when the trigger happens, it doesn't seem to tie up a thread pool thread; at least `ThreadPool.GetAvailableThreads()` doesn't change. Not sure how that works. Also, the specific question here ("Can AsyncController be used to service large number of inactive concurrent requests") isn't answered until you "do some research on how to load test it" is it? :) – bzlm Aug 04 '11 at 07:42