Best approach to Timeout using HttpWebRequest.BeginGetResponse

Question

HttpWebRequest.BeginGetResponse doesn´t respect any Timeout properties from HttpWebRequest(Timeout or ReadWriteTimeout).

I read some approaches to get the same results, but I don't know if it's the best way to do it and if I should use for few calls or I can scale it inside loops(I am doing a webcrawler).

The important thing is, initially my code isn´t async, I just need async because my method should accept a CancellationToken.

My concern is about WaitHandles and ThreadPool.RegisterWaitForSingleObject. It isn´t a daily code then I don´t know if I can have problems in the near future.

private static void HandleCancellation(HttpWebRequest request, IAsyncResult getResponseResult, CancellationToken cancellationToken)
{
    using (WaitHandle requestHandle = getResponseResult.AsyncWaitHandle)
    {
        ThreadPool.RegisterWaitForSingleObject(requestHandle, TimeoutCallback, request, request.Timeout, true);

        //If request finish or cancellation is called
        WaitHandle.WaitAny(new[] {requestHandle, cancellationToken.WaitHandle});
    }

    //If cancellation was called
    if (cancellationToken.IsCancellationRequested)
    {
        request.Abort();
        cancellationToken.ThrowIfCancellationRequested();
    }
}

Calling(again, it isn´t async)

IAsyncResult getResponseResult = request.BeginGetResponse(null, null);

HandleCancellation(request, getResponseResult, cancellationToken);

return (HttpWebResponse)request.EndGetResponse(getResponseResult);

Reference: Better approach in management of multiple WebRequest

What problem is it causing you, do you have lots of async requests using resources waiting for timeout? Why not let it just time-out. — Ben Robinson, May 08 '12 at 11:44
Where should I use TPL? I am using Tasks to call this method, but not inside the method — Felipe Pessoto, May 08 '12 at 11:46
@BenRobinson my webcrawler let me configure how many simultaneous download I do. If I let it timeout, when I call the cancellation token I need to wait all downloads timeout — Felipe Pessoto, May 08 '12 at 11:49

score 3 · Answer 1 · answered May 10 '12 at 03:51

3

The MSDN documentation for BeginGetResponse has a very good example of how to handle timeouts. It worked quite well for me in my Web crawler.

answered May 10 '12 at 03:51

Jim Mischel

131,090
20
188
351

I am doing something similar, but I am using WaitHandle.WaitAny instead of ManualResetEvent. It allows a cleaner code – Felipe Pessoto May 10 '12 at 13:01

Best approach to Timeout using HttpWebRequest.BeginGetResponse

1 Answers1

Linked