Async HttpWebRequest with no wait from within a web application

Question

In my web application (ASP.NET) I have a block of code that uses HttpWebRequest to make a call to a REST service and continue execution. Right now it's taking longer than I would like to complete the full web request. The thing is that what the REST service returns isn't useful. Ideally I would like to send an Async web request to the REST service and then NOT wait for a response. The problem is that I've tried it out using

request.BeginGetResponse(New AsyncCallback(AddressOf myFunc), Nothing)

To start an async request and instead of NOT waiting (which I would assume would be the default behavior of an async request) it continuously executes the callback function before executing the next line of code after BeginGetResponse.

I'm suspecting that ASP.NET may convert it to a sync request when it's within a web application. I'm led to believe this because there's a IAsyncResult result object that is passed into the callback function and when I examine its CompletedSynchronously property it's always set to true.

Does anyone know if it's possible to do an async HttpWebRequest (with no wait) from within an ASP.NET web application or is it always converted to a synchronous request?

score 32 · Answer 1 · edited May 16 '14 at 20:16

32

HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(myUrl);
//set up web request...
ThreadPool.QueueUserWorkItem(o=>{ myRequest.GetResponse(); });

Also known as Fire-and-Forget.

edited May 16 '14 at 20:16

Dan Atkinson

11,391
14
81
114

answered Feb 01 '10 at 17:00

Bryan Batchelder

3,627
21
17

`ThreadPool` that will come in use more often now that I now about it :) That makes fire and forget so simple now... +1 – Piotr Kula May 06 '13 at 15:20
8

Update, a more modern approach for .NET 4.0 may be Task.Factory.StartNew(() => myRequest.GetResponse()); – RJB Feb 04 '14 at 20:03
4

Or even Task.Run(() => ...) – Erti-Chris Eelmaa Jul 12 '15 at 06:48

score 18 · Accepted Answer · answered Feb 01 '10 at 16:52

18

you are probably looking at "fire and forget" pattern. Here are some links.

http://weblogs.asp.net/albertpascual/archive/2009/05/14/fire-and-forget-class-for-asp-net.aspx

http://haacked.com/archive/2009/01/09/asynchronous-fire-and-forget-with-lambdas.aspx

http://www.eggheadcafe.com/articles/20060727.asp

hope this helps

answered Feb 01 '10 at 16:52

ram

11,468
16
63
89

1

This is awesome!! Especially how that haacked.com blog cuts it down to one line of code. I may just end up using this pattern in other parts of my app as well. – Adam Feb 01 '10 at 17:45
I'd want to make absolutely sure that ThreadPool.QueueUserWorkItem was safe in IIS. (Related http://stackoverflow.com/questions/1325718/using-threadpool-queueuserworkitem-in-asp-net-in-a-high-traffic-scenario) – Perhentian Oct 26 '11 at 15:46

score 14 · Answer 3 · edited May 23 '17 at 12:02

(Note: I started to enter this as a comment, but as I gained some extra knowledge while researching, it grew big enough for an answer)

1. ThreadPool.QueueUserWorkItem

The first two links that @ram provides, as well as @Bryan Batchelders answer make use of the controversial ThreadPool.QueueUserWorkItem. It's controversial within the context of ASP.NET because uncareful use may starve your thread pool, as @Perhentian link shows.

2. BeginInvoke

Then I had a look at @ram's third link, which makes use of BeginInvoke. In its essence, this just seems to tell some code to run on another thread too. So no resolution here.

3. BeginGetResponse

Now back to @Perhentians link. It states how BeginGetResponse is somewhat different from an actual thread, because it uses IO Completion Ports (IOPCs). So what you're actually looking for, is a solution which still uses these IOPCs without creating an extra thread.

Now, in order to see how .NET does this, I tried to dig into HttpWebRequest.BeginGetResponse, which is really a rag of internal calls. It goes like:

HttpWebRequest.BeginGetResponse
ServicePoint.SubmitRequest
Connection.SubmitRequest
Two options:
- If the connection is clear: Connection.StartRequest
- Otherwise: Connection.WaitList.Add(request)

A. WaitList

First lets consider option 2: the said WaitList gets processed when a connection is done with a previous request. Taking a look at the ConnectStream involved in this whole chain shows a ThreadPool.QueueUserWorkItem with the remark:

// otherwise we queue a work item to parse the chunk
// Consider: Will we have an issue of thread dying off
// if we make a IO Read from that thread???

So, at least in some fallback scenario's, threads can still inadvertently get spawned by the framework, by just using BeginGetResponse!

B. Connection.StartRequest

Now, we still have the scenario where the connection is clear. The callback is installed by System.Net.Sockets.Socket.BeginConnect and actually invoked by BeginAccept. Some more digging reveals a call to ThreadPool.UnsafeRegisterWaitForSingleObject whose result is used to wait for.

Conclusion

Finally, we can tell what's actually going on when doing a BeginGetResponse:

// 1. Connecting a socket
UnsafeNclNativeMethods.OSSOCK.WSAConnect(m_handle)

// 2. Registering the callback
m_RegisteredWait = ThreadPool.UnsafeRegisterWaitForSingleObject(m_AsyncEvent, s_RegisteredWaitCallback, this, Timeout.Infinite, true);

// 3. Waiting for the socket to complete
UnsafeNclNativeMethods.OSSOCK.WSAEventSelect(m_handle, m_AsyncEvent.SafeWaitHandle, blockEventBits);

Note the Timeout.Infinite. I'm still investigating whether it is possible to get the socket running a codepath that does not perform the wait, but for now, it looks impossible to me.

As goes for my conclusion: there seems to be no easy way to use IOCPs in a Fire-And-Forget scenario. So unless you can get the above socket to not wait for completion on BeginAccept, you're stuck with ThreadPool.