3

I'm sending an HTTPWebRequest to a 3rd party with the code below. The response takes between 2 and 22 seconds to come back. The 3rd party claims that once they receive it, they are sending back a response immediately, and that none of their other partners are reporting any delays (but I'm not sure I believe them -- they've lied before).

HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.example.com");
request.Timeout = 38000;
request.Method = "POST";
request.ContentType = "text/xml";
StreamWriter streamOut = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(XMLToSend);     // XMLToSend is just a string that is maybe 1kb in size
streamOut.Close();
HttpWebResponse resp = null;
resp = (HttpWebResponse)request.GetResponse();      // This line takes between 2 and 22 seconds to return.
StreamReader responseReader = new StreamReader(resp.GetResponseStream(), Encoding.UTF8);
Response = responseReader.ReadToEnd();      // Response is merely a string to hold the response.

Is there any reason that the code above would just...pause? The code is running in a very solid hosting provider (Rackspace Intensive Segment), and the machine it is on isn't being used for anything else. I'm merely testing some code that we are about to put into production. So, it's not that the machine is taxed, and given that it is Rackspace and we are paying a boatload, I doubt it is their network either.

I'm just trying to make sure that my code is as fast as possible, and that I'm not doing anything stupid, because in a few weeks, this code will be ramped up to run 20,000 requests to this 3rd part every hour.

Kenster
  • 23,465
  • 21
  • 80
  • 106
Matt Dawdy
  • 19,247
  • 18
  • 66
  • 91
  • Do you have any proxies between you and the target server? If the server's response time isn't the delay, maybe the delay is actually in establishing the connection. – Jacob May 05 '11 at 20:23
  • Jacob, I don't believe so. At least not on my side. I bet they do on their side, though, and that's what I'm trying to prove to them. – Matt Dawdy May 05 '11 at 20:35
  • 1
    Nothing can make a good day go bad like having to prove someone else's problem. – Chris Marisic May 05 '11 at 20:47

3 Answers3

4

Try doing a flush before you close. 

streamOut.Flush();
streamOut.Close();

Also download microsoft network monitor to see for certain if the hold up is you or them, you can download it here...

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=983b941d-06cb-4658-b7f6-3088333d062f&displaylang=en

Dean North
  • 3,741
  • 2
  • 29
  • 30
  • @Dean North -- I downloaded that, and ran it. Drinking from a firehose. Every entry I see that looks like it pertains to me is under process name w3wp.exe (makes sense since my code is in a .ashx file and is started by a call to it). And I do see a jump in time between those entries of 16 seconds, which my logging indicates is the time that the call took to complete. Is there anything specific I should be looking for maybe? – Matt Dawdy May 05 '11 at 20:46
  • Are you able to make the same post to "https://www.xxx.com" via a simple html form to test the speed completely outside of your code? If that also has a 16 second lag then it points the finger of blame directly at the 3rd party. – Dean North May 05 '11 at 20:49
  • Yes. 22 seconds just now. I personally am sure it is them. – Matt Dawdy May 05 '11 at 21:36
  • It has definitely ruled out your code as the cause of the problem. All that is left is your connection to their server, try running your html file from a different location. If that is also slow then I would send them the example html page that shows the slowdown and a screen shot of your network monitor log that shows the delay and push it back over to them. – Dean North May 05 '11 at 21:44
  • The original question as I asked it was if my code was causing the delay to the 3rd party server, and you and RubbleFord both helped me out with some code suggestions. He had a couple more that I implemented, too, so I gave him the "Answer" checkmark. But I upvoted your answer of course, and I just wanted to say thanks for all of your help. – Matt Dawdy May 06 '11 at 15:24
  • Fair enough, glad I could help :) – Dean North May 06 '11 at 16:39
2

There is a few things that I would do:

  1. I would profile the code above and get some definitive timings.
  2. Implement the using statements in order to dispose of resources correctly.
  3. Write the code in an async style there's going to be an awful lot of IO wait once its ramped.
RubbleFord
  • 7,456
  • 9
  • 50
  • 80
  • Answering your points: 1 -- I've done this. I've verified that, down to the thousandth of a second (per log4net) that the line that takes the longest is, in fact, the resp = (HttpWebResponse)request.GetResponse(); line. 2 -- I'll do that (in addition to the suggestion from Dean North above). 3 -- I may or may not be doing this -- this code exists in a method in a class that I am running from a .ashx handler, and I'm running it asynchronously (Task ccTask = Task.Factory.StartNew(cc.RunWebRequest);). Or do you mean something like this: http://stackoverflow.com/questions/202481 – Matt Dawdy May 05 '11 at 20:34
  • Yes as per your link, this will release the thread for other processing until a callback needs to be processed. Otherwise your just blocking inside a task. – RubbleFord May 05 '11 at 20:42
  • Also have you lifted the max connections to an ip in your web config / app config. A loosely remember a figure of 2 is the default. – RubbleFord May 05 '11 at 20:43
  • Quick question for you -- I think I've got it 99% of the way there (asynchonously I mean). But I am getting an error "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host." I'm calling EndGetResponse just fine (I think), and I can make a streamreader from response.GetResponseStream(), but when I do .ReadToEnd() that's when it blows up. Any ideas? Should I just ask a brand new question? – Matt Dawdy May 05 '11 at 21:34
  • I got it to work -- thanks for all of your help. The issue was a stupid one on my part, of course. But now I'm using "using" statements where I can, I'm .Flush() ing the output stream before I close it (and it's wrapped in a using statement), and now I'm using an Asynchronous call to avoid blocking the thread. No I'm going to figure out what you are referring to when you mention the max connections. Thanks for all of your help. – Matt Dawdy May 06 '11 at 15:23
  • It's a config setting, and it allows you to raise the number of connections to an ip address – RubbleFord May 14 '11 at 11:02
1

Can you hit the URL in a regular ole browser? How fast is that?

Can you hit other URL's (not your partner's) in this code? How fast is that?

It is entirely possible you're getting bitten by the 'latency bug' where even an instant response from your partner results in unpredictable delays from your perspective.

Another thought: I noticed the https in your URL. Is it any faster with http?

n8wrl
  • 19,439
  • 4
  • 63
  • 103
  • I can hit it with a browser, but the response is still slow (indicating that it is not my code). I haven't tried getting to other urls. I'll find another server that I can do some sort of test with that is outside of the Rackspace environment. 'Latency bug' -- you mean, where the internet just sucks for no discernible reason sometimes? I can't test with http -- it's always going to be https, and they don't want to spend time on what they consider "my issue". – Matt Dawdy May 05 '11 at 20:38
  • @Matt: by 'latency bug' I really mean what all sits between you and your URL? I guess that means the Internet just sucks, but there are soooo many factors. This is interesting: http://dsoguy.blogspot.com/2007/06/latency-v-throughput.html – n8wrl May 06 '11 at 15:30
  • That's basically what I mean. Thanks for the link -- great analogy. As for what sits between me and the target URL, I honestly don't know that much. Tracert is helpful to a point, then just times out after hop 12. Probably because their hardware is set to ignore it. – Matt Dawdy May 06 '11 at 16:38