14

Question:

I've been told that best practice states that long running http web requests should be turned into shorter asynchronous requests with a mechanism for polling for completion.

Why?

Important Distinction:

I'm working on a web service API. It's not meant to be called by browsers (which would hang on the load) but by rich clients (which call remote services asynchronously anyways) and scripts (which can do the same asynchronous trick)

Motivation:

I'd like to know because I'm trying to make decisions as to when a request should be made asynchronous, what is the cutoff point? I'm working on a web based API that has requests which take anywhere from 0.001 seconds to 400 seconds (and everywhere in between) depending on the request (not the parameters but which actual method they're calling).

I could make everything asynchronous (except for the poll for command completion) but that complicates the work done by API clients (i.e. getting results from requests, polling for completion, etc.)

As far as I know I could also make everything synchronous since the same amount of work is getting done either way so it seems the load will be similar.

Furthermore, all the web services I've used seem to follow a hybrid model so they must be making the decision somehow.

The only way I could really answer this question is to know why that best practice exists.

dkb
  • 4,389
  • 4
  • 36
  • 54
Pace
  • 41,875
  • 13
  • 113
  • 156
  • How long do browsers allow a connection to remain idle before deciding it's timed out? – Will A May 11 '11 at 22:27
  • 2
    @Will "It depends" but the user will likely hit REFRESH REFRESH REFRESH BACK BACK REFRESH (while possibly cursing the site developers) long before that. (I do) –  May 11 '11 at 22:30
  • @pst that's a problem with the lack of loading screens. If a request takes too long return a loading screen at the start and keep the user informed of the progress. – Raynos May 11 '11 at 22:32
  • I'll edit the original question to bring attention to the fact but this is a web service API meant to be called by rich clients, other servers, and scripts, not directly by browsers. – Pace May 11 '11 at 22:47
  • 1
    @Pace if it's not called by rich clients would a TCP socket with a well defined communication protocol be better? – Raynos May 11 '11 at 23:08
  • +1 for raynos comment. oace - are you sure HTTP is your friend? – jon_darkstar May 12 '11 at 05:38
  • Hah, I'm fairly sure HTTP is NOT my friend. Not even close. But I AM communicating with rich clients (a.k.a. Flex). I suppose my wording was somewhat unclear. – Pace May 12 '11 at 12:48
  • @pst but theres no way to know how long an HTTP request will take is there? all you can give them is a spinning icon or a "loading..." with no real progress indicator – ScottC Apr 06 '15 at 17:45
  • The longer the request takes, the highier is the probability of hardware/software problems end it prematurely. – Paulo Morgado Jan 20 '16 at 09:29
  • Take a look at the [CQRS pattern](https://en.wikipedia.org/wiki/Command–query_separation "Command-Query separation"). – Paulo Morgado Jan 20 '16 at 09:30

1 Answers1

20

Asynchronous APIs do not block. Every synchronous call waits and blocks for your results to come back. This is just a sleeping thread and wasted computation.

If you need something to happen, send of an asynchronous request and do further computation when the request returns. This means your thread sits idle and can pick up other work.

Asynchronous requests is the way to scale to thousands of concurrent users.

but that complicates the work done by API clients

This is just a matter of API design. Generally you can call your web API with a callback to handle this. No polling is required.

WebService.Call("someMethod" (data) -> {
   // do something when data returns.
});
Raynos
  • 166,823
  • 56
  • 351
  • 396
  • How is a sleeping thread wasted computation? Also, clients can always wrap a call asynchronously if they wanted to. This doesn't help a server scale since the server is still doing the work. The server can communicate with other servers asynchronously to scale if it has to but this doesn't mandate that client-server communication be asynchronous. – Pace May 11 '11 at 22:46
  • @Pace there's a limit to the amount threads you can have. Every thread sleeping rather then idling is a wasted thread. It helps clients scale because they do not need to wait on your server to finish computing. – Raynos May 11 '11 at 22:53
  • For an asynchronous server you'd have two threads for every request. One thread doing the actual work and another thread that's responding to the polling, I don't see how an asynchronous model reduces the # of threads on the server. – Pace May 11 '11 at 22:58
  • @Pace an asynchronous server can reduce this issue by being based on an eventloop. Look at such examples as nginx and node.js – Raynos May 11 '11 at 23:00
  • That's a valid argument and could be considered an answer to my question. Technically I wouldn't get that advantage unless I moved to an event-loop based server (using Tomcat right now) so I'll wait and see what other responses come back and mark this correct tomorrow. – Pace May 11 '11 at 23:06