What is a good benchmark test for load testing a production system running IIS?

Question

I've noticed a few posts regarding stress testing IIS, but I'm more interested in knowing a good way to establish a bench mark of what my physical web server can handle.

I'm working on a production system right now that seems to be slowing to a near halt on a daily basis, and no one can seem to figure out what is causing the issue. This is your standard N-tier set up (client, web server, db server). I have created some simple ASP.NET pages that can do a few simple things that I'm using to establish a benchmark for stress testing that I can later user when comparing with the production system that is already in place.

I have already built a sample aspx page that simply returns back some web server statistics about the box utilizations, nothing heavy, maybe a few lines of code. I've also created a simple web service (asmx) that will test a DB connection given the correct id, again nothing heavy just opening a db connection and a sample query that doesn't really do anything.

What I would like to know is, what is a good stress level that the following tests should operate under? In particular, I'm using Microsoft's Web Application Stress Tool, and I'm curious what most people would expect the Stress Level and the Stress Multiplier acceptable ranges should be. I already know at what point the web server starts buckling, its around 20 Stress Level and 10 Stress Multiplier give or take, which results in about ~275 page requests / sec. I'm trying to find out if that number is way too low, and if that's some sort of indication that there's an issue between the client and the web server or a hardware issue of some sort, or if it's more of an indication that there's nothing wrong and that the system is just too heavy.

UPDATE: Since I originally wrote this post the code has been moved to a server running Windows 2003 utilizing IIS 7.

What I'd like to know now is if I set up a relatively simple test, such as a basic index.html file that just serves up a small page, what would you expect the "best" page requests / sec I could achieve from a stress test?

Answer 1

What I'd like to know now is if I set up a relatively simple test, such as a basic index.html file that just serves up a small page, what would you expect the "best" page requests / sec I could achieve from a stress test?

I cannot talk about Microsoft's Web Application Stress Tool, but it should be possible to thrash a static html page hosted on IIS until either the network between the test machine and the server floods or the test server runs out of CPU to create and compile the tests. All ISS would do is send out the same cached response every time.

To get a good profile of how your site is working, you do need to build up a good representation of the actual traffic hitting the site. Once you have the modelled, you can beging scaling up the numbers of people using it to explore what pages are slowest and what server resources are first against the wall.

Either you will have one or more pages whose performance is unacceptable for a given load or some server statistic like disk usage or database performance is looking over utilised.

Unless you are testing on a site that is hosted on the same architecture as your production site, it is hard to make architecture recommendations.

With underperforming pages it is easier to make improvements as the code is available to analyse.

I recommend using Visual Studio Ultimate edition to load test if you can get it. It also gives some ASP.NET profiling tools to get to grips with sections of code and database calls.