ASP.NET: HttpModule performance

Question

I've implemented an HttpModule that intercepts the Response stream of every request and runs a half dozen to a dozen Regex.Replace()s on each text/html-typed response. I'm concerned about how much of a performance hit I'm incurring here. What's a good way to find out? I want to compare speed with and without this HttpModule running.

Answer 1

I've a few of these that hook into the Response.Filter stream pipeline to provide resource file integration, JS/CSS packing and rewriting of static files to absolute paths.

As long as you test your regexes in RegexBuddy for speed over a few million iterations, ensure you use RegexOptions.Compiled, and remember that often the quickest and most efficient technique is to use a regex to broadly identify matches and then use C# to hone that to exactly what you need.

Make sure you're also caching and configuration that you rely upon.

We've had a lot of success with this.

Answer 2

Http module is just common piece of code, so you can measure time of execution of this particular regex replace stuff. It is enough. Have a set of typical response streams as input of your stress test and measure executing of the replace using Stopwatch class. Consider also RegexOptions.Compiled switch.

Answer 3

Here are a few ideas:

1. Add some Windows performance counters, and use them to measure and report average timing data. You might also increment a counter only if the time measurement exceeds a certain threshold. and
2. Use tracing combined with Failed Request Tracing to collect and report timing data. You can also trigger FRT reports only if page execution time exceeds a threshold.
3. Write a unit test that uses the Windows OS clock to measure how long your code takes to execute.
4. Add a flag to your code that you can turn on or off with a test page to enable or disable your regex code, to allow easy A/B testing.
5. Use a load test tool like WCAT to see how many page requests per second you can process with and without the code enabled.

Answer 4

I recently had to do some pef tests on an HTTPModule that I wrote and decided to perform a couple of load tests to simulate web traffic and capture the performance times with and without the module configured. It was the only way I could figure to really know the affect of having the module installed.

I would usually do something with Apache Bench (see the following for how to intsall, How to install apache bench on windows 7?), but I had to also use windows authentication. As ab only has basic authentication I it wasn't a fit for me. ab is slick and allows for different request scenarios, so that would be the first place to look. One other thought is you can get a lot of visibility by using glimpse as well.

Being that I couldn't use ab I wrote something custom that will allow for concurrent requests and test different url times.

Below is what I came up with to test the module, hope it helps!

// https://www.nuget.org/packages/RestSharp
using RestSharp;
using RestSharp.Authenticators;
using RestSharp.Authenticators.OAuth;
using RestSharp.Contrib;
using RestSharp.Deserializers;
using RestSharp.Extensions;
using RestSharp.Serializers;
using RestSharp.Validation;

string baseUrl = "http://localhost/";
void Main()
{
    for(var i = 0; i < 10; i++)
    {
        RunTests();
    }

}

private void RunTests()
{
    var sites = new string[] { 
        "/resource/location",
    };

    RunFor(sites);
}
private void RunFor(string[] sites)
{
   RunTest(sites, 1);
   RunTest(sites, 5);
   RunTest(sites, 25);
   RunTest(sites, 50);
   RunTest(sites, 100);
   RunTest(sites, 500);
   RunTest(sites, 1000);


}
private void RunTest(string[] sites, int iterations, string description = "")
{
    var action = GetAction();

    var watch = new Stopwatch(); 
   // Construct started tasks
   Task<bool>[] tasks = new Task<bool>[sites.Count()];
   watch.Start();

   for(int j = 0; j < iterations; j++)
   {
        for (int i = 0; i < sites.Count(); i++)
        {
            tasks[i] = Task<bool>.Factory.StartNew(action, sites[i]);
        }
   }
   try
   {
       Task.WaitAll(tasks);
   }
   catch (AggregateException e)
   {
       Console.WriteLine("\nThe following exceptions have been thrown by WaitAll()");
       for (int j = 0; j < e.InnerExceptions.Count; j++)
       {
           Console.WriteLine("\n-------------------------------------------------\n{0}", e.InnerExceptions[j].ToString());
       }
   }
   finally
   {
    watch.Stop();
    Console.WriteLine("\"{0}|{1}|{2}\", ",sites.Count(), iterations, watch.Elapsed.TotalSeconds);
   }
}
private Func<object, bool>  GetAction()
{
    baseUrl = baseUrl.Trim('/');
    return (object obj) =>
   {
        var str = (string)obj;
        var client = new RestClient(baseUrl);
        client.Authenticator = new NtlmAuthenticator();
        var request = new RestRequest(str, Method.GET);
        request.AddHeader("Accept", "text/html");
        var response = client.Execute(request);     
        return (response != null);
   };
}