HttpWebRequest times out on second call

Question

Why does the following code Timeout the second (and subsequent) time it is run?

The code hangs at:

using (Stream objStream = request.GetResponse().GetResponseStream())

and then causes a WebException saying that the request has timed out.

I have tried this with a WebRequest and HttpWebRequest

Edit: It seems the code is falling over in request.GetResponse()

Edit: This post suggests it may be a GC issue --> http://www.vbforums.com/showthread.php?t=610043 - as per this post the issue is mitigated if Fiddler is open in the background.

The server is there and available for requests.

    private string GetQLMResponse(string URL)
    {
        HttpWebRequest request = WebRequest.Create(URL) as HttpWebRequest;
        request.Credentials = new NetworkCredential(Settings.Default.LicenseUser, Settings.Default.LicensePassword);
        request.KeepAlive = false;
        request.Timeout = 5000;
        request.Proxy = null;

        // Read stream
        string responseString = String.Empty;
        try
        {
            using (var response = request.GetResponse())
            {
                using (Stream objStream = response.GetResponseStream())
                {
                    using (StreamReader objReader = new StreamReader(objStream))
                    {
                        responseString = objReader.ReadToEnd();
                        objReader.Close();
                    }
                    objStream.Flush();
                    objStream.Close();
                }
                response.Close();
            }
        }
        catch (WebException ex)
        {
            throw new LicenseServerUnavailableException();
        }
        finally
        {
            request.Abort();
            request = null;
            GC.Collect();
        }
        return responseString;
    }

Thrown WebException is:

{"The operation has timed out"} [System.Net.WebException]: {"The operation has timed out"} Data: {System.Collections.ListDictionaryInternal} HelpLink: null InnerException: null Message: "The operation has timed out" Source: "System" StackTrace: " at System.Net.HttpWebRequest.GetResponse()\r\n at IQX.Licensing.License.GetQLMResponse(String URL) in C:\Users\jd\SVN\jd\Products\Development\JAD.Licensing\JAD.Licensing\License.cs:line 373" TargetSite: {System.Net.WebResponse GetResponse()}

---

Update: OK So the following code now works. The servicePoint was setting the timeout to be near 4 minutes. Changing ServicePoint.ConnectionLeaseTimeout on the request object means that the request is now destroyed after 5000ms. Thanks to all for your help and also to these 2 pages:

1. http://blogs.msdn.com/b/adarshk/archive/2005/01/02/345411.aspx

2. http://msdn.microsoft.com/en-us/library/6hszazfz(v=VS.80).aspx

   private string GetQLMResponse(string URL)
   {
       HttpWebRequest request = WebRequest.Create(URL) as HttpWebRequest;
       request.Credentials = new NetworkCredential(Settings.Default.LicenseUser, Settings.Default.LicensePassword);
       request.KeepAlive = false;
       request.Timeout = 5000;
       request.Proxy = null;
   
       request.ServicePoint.ConnectionLeaseTimeout = 5000;
       request.ServicePoint.MaxIdleTime = 5000;
   
       // Read stream
       string responseString = String.Empty;
       try
       {
           using (WebResponse response = request.GetResponse())
           {
               using (Stream objStream = response.GetResponseStream())
               {
                   using (StreamReader objReader = new StreamReader(objStream))
                   {
                       responseString = objReader.ReadToEnd();
                       objReader.Close();
                   }
                   objStream.Flush();
                   objStream.Close();
               }
               response.Close();
           }
       }
       catch (WebException ex)
       {
           throw new LicenseServerUnavailableException();
       }
       finally
       {
           request.Abort();
       }
       return responseString;
   }
   

Answer 1

On the heels of the previous answers, I wanted to add a couple more things. By default HttpWebRequest allows only 2 connections to the same host (this is HTTP 1.1 "niceness"),

Yes, it can be overriden, no I won't tell you how in this question, you have to ask another one :) I think you ought to look at this post.

I think that you are still not quite disposing of all your resources connected with the HttpWebRequest, so the connection pooling comes into play and that's the problem. I wouldn't try to fight the 2 connections per server rule, unless you really have to.

As one of the posters above noted, Fiddler is doing you a bit of a disservice in this case.

I'd add a nice finally {} clause after your catch and make sure that as the above post notes, all streams are flushed, closed and references to the request object are set to null.

Please let us know if this helps.

Answer 2

The WebResponse obtained by request.GetReponse() MUST be disposed properly. Try this (removing request.Abort() and GC.Collect() calls):

using (var wresponse = request.GetResponse())
{
   using (Stream objStream = wresponse.GetResponseStream())
   {
        // ...
   }
}

Edit: Since it still does not work, I suggest you to test this with an empty windows application. This way, you could isolate app.config problems or maximum concurrent calls per host* (are you using other webrequest object somewhere else in your application to this host; which webresponse are not disposed properly?).

Hope this solve your problem, I am out of ideas!

• See Jon Skeet's answer here.

Answer 3

As you have stated, running fiddler in the background would mitigate the issue. This is because fiddler force closes any responses. Extending on the above post from Sam B I would ensure that the response is closed like so:

using (var wresponse = request.GetResponse())
{
   using (Stream objStream = wresponse.GetResponseStream())
   {
        // ...
   } 
   wresponse.close();
}

Also it may be worth setting the proxy to null like so:

 request.Proxy = Null;

As the .NET framework will go out searching for a proxy unless you explicitly do this. When fiddler is running this effect would be mitigated as fiddlers proxy would be found directly.

Answer 4

ran into the same problem with timeouts on subsequent requests to the server despite disposing/flushing/closing everything properly. try flushing your connection group, worked for me:

myRequest.ServicePoint.CloseConnectionGroup(myRequest.ConnectionGroupName);

also, ensure you're not inadvertently creating other HttpWebRequest/Request objects elsewhere in your application that aren't being properly terminated/disposed, as this will increase the numbers of connections in the Service Point.

Answer 5

I had the same issue, and I resolved it ensuring I call the Abort() method on each of the request object created.

Answer 6

By any chance were you using a test app with the default name of WindowsFormsAppN? I had the same problem that I spent a week debugging because it worked in my production code but not in a simple test solution I was building. In the end, I determined this behavior was unique to using the default solution name instead of a properly named solution.

Edit: I discovered my issue was related to using BitDefender as my AV software. The WindowsFormsAppN programs were all blocked.

Answer 7

I read the post tagged as answer above and it didn't really work for me above. I ended up rewriting the code using the using tags and added a CloseConnectionGroup segment someone else recommended here. Ultimately this is what the code looks like:

System.Net.HttpWebRequest webRequest = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(imageUrl);
                webRequest.AllowWriteStreamBuffering = true;
                webRequest.Timeout = 30000;
                webRequest.ServicePoint.ConnectionLeaseTimeout = 5000;
                webRequest.ServicePoint.MaxIdleTime = 5000;

                using (System.Net.WebResponse webResponse = webRequest.GetResponse())
                {

                    using (System.IO.Stream stream = webResponse.GetResponseStream())
                    {
                        image = System.Drawing.Image.FromStream(stream);
                    }
                }

                webRequest.ServicePoint.CloseConnectionGroup(webRequest.ConnectionGroupName);
                webRequest = null; 
            }

Answer 8

Done with me when add this

myRequest.ServicePoint.CloseConnectionGroup(myRequest.ConnectionGroupName);

Answer 9

for anyone looking for second time timeouts in vb.net: make sure you have been using:

1. response.close()

2. response.dispose()

after every time you have used your webresponse.

Answer 10

I've set http time out to 10 minutes and it worked for me.

Setting to timeout=infinite was taking more time and my program was going in hung mode.