Using worker threads or ThreadPool threads for async I/O in a Web application

Question

I infer that since an ASP.NET web application is granted a fixed number of worker threads and I/O threads, which is governed by the <processModel> settings in the web.config file, thus, for a large website with a number of users, there is no particular benefit of spinning new worker threads or using the threads out of the .NET ThreadPool to do tasks like logging errors or sending email, etc.

In other words, suppose that in my web application, for every request that comes in, the request handler executes 3 tasks in a row, namely Task A, followed by Task B, followed by Task C, and then returns the result, which is an HTML page.

However, out of the three tasks, Task B does not have any bearing on the resultant HTML. It might be a task such as logging a piece of text in a log file.

Then, if I moved Task B to another worker thread, while the CurrentThread, which is also a worker thread from the ASP.NET pool of worker threads, will return faster and will result in a better response time for my application's user, the number of concurrent users that my application can service will be affected as Task B will need a new worker thread allocation from the same ASP.NET pool of worker threads.

Therefore, is my understand correct that in a Web application scenario: forking out new threads will affect performance positively but will affect scalability adversely. Thus, we must only write multi-threaded server side code if we have the luxury of scaling out by buying lots of hardware?

If not, then we are only trading off one for the other?

Answer 1

The core of your question seems to be: "Will using other threads to process secondary concerns asynchronously, hinder my ability to scale vertical performance? ie. number of connections each web server can handle at once."

There can generally be no more requests actively being processed, than there are threads. There are 100 threads allocated into the .Net thread pool for every virtual CPU core (logical processor). You can test this on your machine with the following call ThreadPool.GetAvailableThreads Method.

In my experience the short answer is "yes", but this is only true if the web server's CPU is the bottleneck of your application's traffic load. So if the web server is waiting on the database, or waiting for API calls to return, you may max out your other resources before you max out your CPU.

The recommended solution when attempting to scale the processing of independent processing tasks, is to use a queue, such as Microsoft Message Queue, or RabbitMQ. Then you can separately dedicate as many threads on a "services" machine as necessary to complete those request independent tasks, while maximizing good use of your web server resources.

Using a queue for offline processing is also easy to scale. When you are small, you can run all of these services on the same box, but when you get to the point where more processing power is needed, you can move that service to another machine or as many machines as required.

This linked question and answer is somewhat similar.