1

I'm using a 3rd party library that makes a number of http calls. By decompiling the code, I've determined that it is creating and using raw HttpWebRequest's, all going to a single URL. The issue is that some of the requests don't get closed properly. After some time, all new HttpWebRequest's block forever when the library calls GetRequestStream()* on them. I've determined this blocking is due to the ConnectionLimit on the ServicePoint for that particular host, which has the default value of 2. In other words, the library has opened 2 requests, and then tries to open a 3rd, which blocks.

I want to protect against this blocking. The library is fairly resilient and will reconnect itself, so it's okay if I kill the existing connections it has made. The problem is that I don't have access to any of the HttpWebRequest or HttpWebResponses this library makes. However I do know the URL it accesses and therefore I can access the ServicePoint for it.

var sp = ServicePointManager.FindServicePoint(new Uri("http://UrlThatIKnowAbout.com"));

(Note: KeepAlive is enabled on these HttpWebRequests)

dan
  • 9,712
  • 6
  • 49
  • 62
  • 1
    If you solved your problem, please write a helpful explanation of the steps you took for others who may find themselves in a similar situation in the future. Thank you. – Sampson Nov 23 '11 at 04:49
  • @jonathan I made an educated guess and proposed 3 steps in my question as the solution, and it ended up working. What's the confusion? – dan Nov 23 '11 at 20:10
  • Your answer couldn't stand on its own. You said "I did this, and it worked." The reader is left wondering what "this" and "it" refer to, as was evident in the comment left before deletion. Please feel free to write out an answer that is able to stand on its own. Thanks. – Sampson Nov 23 '11 at 21:27
  • @Dan, I am also interested in how the TCP connection is established - however I couldn't find reference when I am looking at the assembly in ilsphy? do you have any luck? if yes, can you give me some advise? Basically I am trying to understand samething as yours 'I was interested to learn that GetRequestStream() establishes a TCP connection with the remote server, while GetResponse() actually sends the request' - how http webrequest creates tcp connections, manages them? – Dreamer Apr 02 '14 at 01:55
  • I think I understand details little better I guess (for details): http://stackoverflow.com/questions/22800216/how-and-where-the-tcp-connection-has-been-created-in-httpwebrequest-and-how-is – Dreamer Apr 02 '14 at 19:28
  • I was interested to learn that `GetRequestStream()` establishes a TCP connection with the remote server, while `GetResponse()` actually sends the request. – dan May 22 '14 at 20:59

1 Answers1

1

This worked, though I'm not sure it's the best way to solve the problem.

  1. Get the service point object for the url
    var sp = ServicePointManager.FindServicePoint(new Uri("http://UrlThatIKnowAbout.com"));
  2. Increase the ConnectionLimit to int.MaxValue
  3. Create a background thread that periodically checks the ConnectionCount on the service point. If it goes above 5, call CloseConnectionGroup()
  4. Set MaxIdleTime to 1 hour (instead of default)

Setting the ConnectionLimit should prevent the blocking. The monitor thread will ensure that too many connections are never active at the same time. Setting MaxIdleTime should serve as a fall back.

dan
  • 9,712
  • 6
  • 49
  • 62