Occasionally while crawling, I get a VERY slow response.
It is not slow to connect, so timeout doesn't work. Its the host speed going 3-4 Bytes/second.
Is there a way to kill this request if it takes more than 10 seconds, like a timeout?
Asked
Active
Viewed 455 times
0
-
Probably the best way is to use an asynchronous request and then Abourt() it after a certain amount of time. http://stackoverflow.com/questions/5520122/canceling-async-httpwebrequests – anon Jul 16 '11 at 00:48
2 Answers
2
What you want to do is start your crawl process on its own thread that times out after a specified time. Have a look at BeginGetResponse() on the HttpWebRequest class, which starts a thread and calls a specified function when it completes. You can add a timeout mechanism to this method, as shown under the Example section on the following page: http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse(v=VS.71).aspx.
Sorry, I don't have any code handy to show how this is done, but the example on that page appears complete.
Thorin
- 626
- 4
- 11
0
If your HttpWebRequest is in a thread you schould be able to kill the request by calling Abort on the thread itself... I know this is perhaps "overkill"... perhaps someone else has better idea...
Yahia
- 69,653
- 9
- 115
- 144