HttpWebRequest, Keep-Alive as Fiddler does?

Question

Using HttpWebRequest, I'm trying to query a secured (negotiate) url behind a load balancing setup in round-robin mode (two IIS 7.5 servers). Seems simple enough, but I have some problems.

The first anonymous request goes on one server and the negotiate part goes on the other. The problem is that it takes about six seconds between these two requests, so it is way too long. Trying to diagnose the delay, I realized that, going through Fiddler's proxy, all the requests went on the same server, so it took less than one second total. If I disable Fiddlers option "reuse server connections", then my requests have the same behavior as without Fiddler and it takes forever.

Googling this, I ended up on this link: http://fiddler2.com/blog/blog/2013/02/28/help!-running-fiddler-fixes-my-app-

I know that Fiddler is using sockets and its own connection pool, but is there a way to reproduce the same behavior using .NET HttpWebRequest so that my requests (anonymous and negotiate) will reuse connections and end up on the same server?

Here is a quick test that takes about 70 seconds to complete without Fiddler, and about 2 seconds going through Fiddler...

Also, please note that it isn't a Proxy detection delay and that sticky session are disabled on the nlb.

    public void Main(string[] args)
    {
        int i = 0;

        while (i < 10)
        {
            HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("http://nlb/service.asmx");
            HttpWebResponse response;

            wr.KeepAlive = true;
            wr.UseDefaultCredentials = true;
            response = (HttpWebResponse)wr.GetResponse();

            using (StreamReader sr = new StreamReader(response.GetResponseStream()))
            {
                Console.WriteLine(sr.ReadToEnd());
            }

            response.Close();

            i++;
        }
    }

This is another proof that Fiddler is plain awesome!

Thanks for any advice.

Just shooting in the dark here - did you try using the new HttpClient API instead? Also, this question might help: http://stackoverflow.com/questions/17894739/connection-keep-alive-not-working-with-system-net-http-httpclient-on-certain-hos — sinelaw, Nov 11 '13 at 04:51
@sinelaw Unfortunately, I can't use 4.5 in my development environment, but I hope I could accomplish the same behavior with HttpWebRequest since .Net seems to always use WebRequest when it comes to http (no matter what wrapper is used). — ChG, Nov 11 '13 at 13:36
You don't need .Net 4.5 - you can use the [httpclient nuget package](http://www.nuget.org/packages/Microsoft.Net.Http) — sinelaw, Nov 11 '13 at 14:52
@sinelaw Unfortunately I obtain the same result with HttpClient. Thanks for the suggestion. — ChG, Nov 12 '13 at 01:16
If you can email me a SAZ file from Fiddler (use Help > Send Feedback) I'd be happy to have a look. You also might consider enabling System.NET logging in your app.exe.config file; the logs *should* show why connections aren't getting reused as expected. — EricLaw, Nov 12 '13 at 04:28
Have you seen [this post](https://fiddler2.com/blog/blog/2013/02/28/help!-running-fiddler-fixes-my-app-)? — shamp00, Dec 17 '13 at 10:21
@shamp00 Yes, as you can read in my question, I ended up on this fiddler's post, hence the question on how to have my connections behave like when I'm proxying through fiddler. — ChG, Jan 05 '14 at 18:27
This may seem obvious, but what happens if you omit the `Response.Close()` line? — theMayer, Jan 27 '14 at 03:01

score 0 · Answer 1 · answered Jan 27 '14 at 03:05

Just a shot here, and maybe it seems too easy -

But the last line of your code is Response.Close(). The documentation prior to .NET 4.5 doesn't say much about this other than it "closes the existing socket connection."

In .NET 4.5 however, this is the documentation:

This method terminates the connection to the client in an abrupt manner and is not intended for normal HTTP request processing.

http://msdn.microsoft.com/en-us/library/system.web.httpresponse.close(v=vs.110).aspx

I'll admit that I don't know some of the subtle differences between .NET 4.5 and the prior versions of HttpResponse; however, I do think that logically, Connection.Close() is not compatible with Keep-Alive; and you could be seeing the behavior of Fiddler intervening (maybe as a bug) to patch over this. Just a theory- needs testing.

Nice theory, but I think the close is not where the problem lies. It's when the request starts - OP says at negotiate, which is when the credentials are exchanged at the start. — Simon Halsey, Jan 27 '14 at 03:21
Good point. I wonder then if maybe Fiddler is being smarter about auth than the HttpClient? — theMayer, Jan 27 '14 at 03:26

HttpWebRequest, Keep-Alive as Fiddler does?

1 Answers1