Spring @RestController invoking large number of requests

Question

I am implementing a REST API using Spring Boot and @RestController with Java 8. One of the controller methods needs to invoke another, 3rd party REST API service. The method is outlined as follows (data format is JSON):

1. Call a 3rd party API method to get a list of candidates (each candidate is represented as an object with some basic information).
2. For each candidate, call another 3rd party API method to get some more detailed information about the candidate.
3. Mash up the results, essentially "enriching" all the candidate objects from the first call.
4. Return the list of enriched candidate objects.

I was planning on using @RestTemplate for all invocations on the 3rd party API. What I am concerned about is that for a large number of candidates (say 500-1000), this is going to become a huge performance bottleneck if implemented in a blocking fashion. I am not quite sure what is the recommended approach for best performance. How can I scale this so multiple users can concurrently access my API?

Answer 1

Your question is VERY broad and I don't think anyone will be able to provide you a detailed answer, but:

1. Your case seems like a perfect use case for RxJava: a stream of candidates that gets transformed by adding additional information from other sources - all done async.

2. I would skip RestTemplate and go for Retrofit2 together with their awesome RxJava integration. See this basic tutorial1 and tutorial2.

3. 1000 records is not much but it depends on the performance of the downstream services as well. You will need to test it for yourself.

4. If you want to know more about RxJava there are some great docs online as well as a new amazing book by Tomasz Nurkiewicz and Ben Christensen.

Good luck!

Answer 2

Rafal gave a great answer already. I would like to add my 2 cents. Your case seems like a perfect use case for Non-Blocking. RxJava is just one implementation of non blocking reactive programming. You can achieve the same thing using different other technology stack as well - like akka or spring boot with netty or undertow.

This is an example of how we can do reactive programming with spring boot. https://spring.io/blog/2016/07/28/reactive-programming-with-spring-5-0-m1 keep in mind that, its still in experimental state. Reactive streams will be built into Java 9

Answer 3

Response to an old thread. Below are some of things that can be used.

1. Use AsyncHttp instead of RestTemplate.
2. Setup a separate thread pool (Executor Service) for making external calls.
3. Call the external API using Java 8 CompletableFuture.supplyAsync(() -> externalCall(),executorService) where possible. For both first 3rd party call and the nested call.
4. Return the mashed up object wrapped with CompletableFuture to the controller.

Answer 4

Your application server will manage a thread pool, so many requests can hit your endpoint at the same time. Unless your thread pool is of size 1, multiple clients will be able to simultaneously use your API.

See here for further explanation: Spring MVC Rest Services - Number of Threads (Controller Instances)

See here for how to an example of how configure thread pool size: Maximum (client request) thread pool size in spring

In order to improve performance when making requests to the external services, I would see if you could make batch requests (multiple candidates at once), to reduce the total number of requests you make.