Asynchronous and synchronous HTTP request on server side, performance comparison

Question

I am trying to figure out the pros and cons of asynchronous and synchronous HTTP request processing. I am using the Dropwizard with Jersey as my framework. The test is comparing the asynchronous and synchronous HTTP request processing, this is my code

@Path("/")
public class RootResource {

    ExecutorService executor;

    public RootResource(int threadPoolSize){
        executor = Executors.newFixedThreadPool(threadPoolSize);
    }

    @GET
    @Path("/sync")
    public String sayHello() throws InterruptedException {
        TimeUnit.SECONDS.sleep(1L);
        return "ok";
    }

    @GET
    @Path("/async")
    public void sayHelloAsync(@Suspended final AsyncResponse asyncResponse) throws Exception {
        executor.submit(() -> {
            try {
                doSomeBusiness();
                asyncResponse.resume("ok");
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        });
    }


    private void doSomeBusiness() throws InterruptedException {
        TimeUnit.SECONDS.sleep(1L);
    }

}

The sync API will run in the worker thread maintained by the Jetty and the async API will mainly run in the customs thread pool. And here is my result by Jmeter

Test 1, 500 Jetty worker thread, /sync endpoint
Test 2, 500 custom thread, /async endpoint

As the result shows, there are no much differences between the two approaches.

My question would be: What's the differences between these two approaches, and which pattern should I use in which scenario?

Related topic : Performance difference between Synchronous HTTP Handler and Asynchronous HTTP Handler

update

I run the test with 10 delays as suggested

sync-500-server-thread

async-500-workerthread

[1] In the "/async" Summary Report, that _min_ value of 7 (milliseconds?) looks suspiciously low. If you repeat the run does the report still show a similarly low value for _min_? [2] You might want to consider replacing one of your tags with **jmeter** to attract a more suitable audience. — skomisa, Apr 29 '19 at 21:35
@skomisa it is because there are some socket error when I run the async test, as you can see, the error rate is 1.08% — Jade Tang, May 01 '19 at 09:35
@user7294900 I rerun the test case with 10 seconds delay, actually, the async way is slightly worse regarding the max latency, I assume this is because it has more context switching here — Jade Tang, May 01 '19 at 10:47

RAJKUMAR NAGARETHINAM · Answer 1 · 2019-05-21T11:43:16.127

The following are my thoughts.

Whether its synchronous or asynchronous request, its nothing related to the performance of HTTP but it related to your application's performance

Synchronous requests will block the application until it receives the response, whereas in asynchronous request basically, you will assign this work in a separate worker thread which will take care of the rest of things. So in asynchronous design, your main thread still can continue its own work.

Let say due to some limitation(not resource limitation of the server) your server can handle a limited number of connections (Basically each and every connection will be handled in a separate thread differs between the server we use). If your server can handle more number of threads than the connections and also if you don't want to return any data as a result of async work you have created, then you can design asynchronous logic. Because you will create a new thread to handle the requested task.

But if you expect results of the operations to be returned in the response nothing will differ.

Are going to Create a new HttpRequest with your client connection thread or from your server ? — RAJKUMAR NAGARETHINAM, May 02 '19 at 09:25
yes, I agree. But in most cases, the main thread has nothing to do but simply waiting for the response from another worker thread. — Jade Tang, May 02 '19 at 16:07

Ori Marko · Answer 2 · 2019-05-01T13:32:22.083

5

You are using @Suspended combined with async which still wait for response

@Suspended will pause/Suspend the current thread until it gets response

If you want to get better performance in async, write a different async method with immediate response using ExecutorService and Future

private ExecutorService executor;
private Future<String> futureResult;
@PostConstruct
public void onCreate() {
    this.executor = Executors.newSingleThreadExecutor();
}
@POST
public Response startTask() {
    futureResult = executor.submit(new ExpensiveTask());
    return Response.status(Status.ACCEPTED).build();
}
@GET
public Response getResult() throws ExecutionException, InterruptedException {
    if (futureResult != null && futureResult.isDone()) {
        return Response.status(Status.OK).entity(futureResult.get()).build();
    } else {
        return Response.status(Status.FORBIDDEN).entity("Try later").build();
    }
}

edited May 01 '19 at 13:32

answered May 01 '19 at 12:59

Ori Marko

56,308
23
131
233

thanks for pointing out the suspend annotation. But as described in the code, the endpoints are supposed to return "Hello" as a business requirement. If I remove the suspend, the client will get HTTP 204 without a response body. – Jade Tang May 01 '19 at 13:53
@JadeTang you can return "Hello" but still use `Future` – Ori Marko May 02 '19 at 09:34
If I remove the suspend, the client will get a HTTP 204 with an empty response – Jade Tang May 02 '19 at 12:12
@JadeTang don't remove suspend, but still use `Future` – Ori Marko May 02 '19 at 12:12

score 3 · Answer 3 · edited May 01 '19 at 11:07

Let's consider the following scenario:

Single Backend system
                    ____________
                   |  System A  |
 HTTP Request -->  |            |
                   |  1.        |
                   |  2.        |
 HTTP Response <-- |            |
                   |____________|

You have one backend system which does some processing based on the request received on a particular order ( operation 1 and then operation 2 ). If you process the request synchronously or asynchronously doesn't really matter, it's the same amount of computation that needs to be done ( maybe some slight variations like you have encountered in your test ).

Now, let's consider a multi-backend scenario:

Multi-Backend System
                        ____________
                       |  System A  |       __________
     HTTP Request -->  |            | -->  |          |
                       |  1.        |      | System B |
                       |            | <--  |__________|
                       |            |       __________  
                       |  2.        | -->  |          |
     HTTP Response <-- |            |      | System C |
                       |____________| <--  |__________|

Still, 2 processing steps required to be done but this time, on each step we will call another back'end system.

SYNC processing:

Call System B
Wait for a response from System B
Call System C
Wait for a response from System C

Total time spent: B + C

ASYNC processing:

Call System B
Go forward since the call is not blocking
Call System C
Go forward since the call is not blocking
Receive a response from System B
Receive a response from System C
Complete the call to the client

Total time spent: max(B, C)

Why max? Since all the calls are non-blocking then you will have to wait just for the slowest back'end to reply.

I think you are confusing with the server side async/sync HTTP request processing with the async/sync HTTP client. If I process the HTTP request in a sync way, I can still use async HTTP client to request the system B and C which give me the response time max(B,C) — Jade Tang, May 01 '19 at 09:34

score 0 · Answer 4 · answered Apr 26 '19 at 08:12

I am writing this from my personal experience writing a Promotions engine using Asynchronous handlers for Sri Lanka's front running Taxi Hailing service, which even competes shoulder to shoulder with Uber.

It is more scale-ability, availability and resource utilization than performance when it comes to using Asynchronous handlers over synchronous handlers.

If you are using synchronous handlers, your maximum concurrent no of requests is defined by the no of threads available to accept new connections after which your service can no longer accept request at all.

Where as if you use asynchronous handlers, the accept thread count has nothing to do with no of concurrent requests you can cater. Thus your service can scale from 100 rps to 1 million rps and can have high availability.

If you are concerned about latency and throughput, you can get good improvements if you are using Non Blocking APIs with Asynchronous Handlers. Non Blocking Socket (NIO), Non Blocking Storage (Mongo DB Reactive, Redis Reactive), Message Queues (Kafka, RabbitMQ) etc.

Where as if you use asynchronous handlers, the accept thread count has nothing to do with no of concurrent requests you can cater. But in this case, you need another thread pool to process your HTTP request, which can not be infinity either. — Jade Tang, Apr 26 '19 at 08:24

score 0 · Answer 5 · answered Jul 29 '21 at 19:09

The main performance advantage of asynchronous programming is that you can reduce the amount of threads on your system. The documentation of Jetty looks like a good reference.

The servlet API (pre 2.5) supports only a synchronous call style, so that any waiting that a servlet needs to do must be with blocking. Unfortunately this means that the thread allocated to the request must be held during that wait along with all its resources: kernel thread, stack memory and often pooled buffers, character converters, EE authentication context, etc. It is wasteful of system resources to hold these resources while waiting. Significantly better scalability and quality of service can be achieved if waiting is done asynchronously.

The code of the question does not tap into this advantage because it still blocks a thread for each request, it is just a different thread from a custom thread pool:

@GET
@Path("/async")
public void sayHelloAsync(@Suspended final AsyncResponse asyncResponse) throws Exception {
    executor.submit(() -> {
        try {
            doSomeBusiness();
            asyncResponse.resume("ok");
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    });
}


private void doSomeBusiness() throws InterruptedException {
    TimeUnit.SECONDS.sleep(1L);
}

Note that this implementation is especially problematic because in addition to blocking a thread, the amount of threads is strictly limited by the thread pool.

If you want to tap into the advantages of asynchronous request handling, you have to reimplement TimeUnit.SECONDS.sleep(1L), so that it does not block any thread.

thanks, but in our case, the TimeUnit.SECONDS.sleep(1L) is actually a MySQL database call, I don't see any MySQL database driver that does not block. — Jade Tang, Jul 30 '21 at 20:26

Asynchronous and synchronous HTTP request on server side, performance comparison

5 Answers5