43

I am developing Rest APIs with Spring Boot which is deployed on AWS Beanstalk. Potentially, the service will be getting hits from thousands of clients every day. Therefore I would like to understand capability of Spring Boot of handling multiple requests.

From what I read in Spring-Boot: Handle multiple requests concurrently and How to have thread safe controller in spring boot, it seems Spring Boot can handle requests concurrently while controller being thread safe.

If two requests are made to the same endpoint for updates at the same time though, does the controller deal with the requests one after another or simultaneously with two threads in parallel? If latter, does each thread has its own entity manager? Is there a way to implement a thread pool to limit the number of threads based on the capacity of the EC2 instance? By the way, how do I decide how big of an instance should I start with based on the estimated volumn of requests?

ddd
  • 4,665
  • 14
  • 69
  • 125

3 Answers3

35

Yes, Spring boot can handle simultaneously requests! If your servlet container is tomcat under the hood, it can handle 200 simultaneous requests.

Below snippet highlights the point, but refer original spring boot source

@ConfigurationProperties(prefix = "server", ignoreUnknownFields = true)
public class ServerProperties {
  public static class Tomcat {
     public static class Threads {
       private int max = 200; // Maximum amount of worker threads
     }
  }
}

However, you can override this value by adding server.tomcat.threads.max to your application.properties or application.yml.

Spring will manage a pool of connections and handle the distribution of entity managers (according to the minimum and maximum of connections you specify in your properties). I believe you can read more about it here: When are connections returned to the connection pool with Spring JPA (Hibernate) Entity Manager?

srk
  • 4,857
  • 12
  • 65
  • 109
Felipe Mariano
  • 564
  • 5
  • 8
  • I am deploying the application as war which does not use embeded tomcat. Does this `server.tomcat.max-threads` still work? – ddd Oct 23 '17 at 16:02
  • No. Then you have to configure your servlet container separately. – dunni Oct 23 '17 at 16:03
  • Oh, in this case, max-threads won't work, but you can change the number of concurrent connections in your Elastic Beanstalk! Just enter in your Worker, go to `Configuration > Worker Configuration` and change the `HTTP Connections` field. – Felipe Mariano Oct 23 '17 at 16:06
  • @FelipeMariano Can't seem to find where the `worker Configuration` is. Is it in Beanstalk console where I define `SERVER_PORT` and other environment variables? – ddd Oct 23 '17 at 18:39
  • Oh! Here I'm using some Worker's Environments and I can edit the number of connections in the section Worker Configuration (Here I define the environment variables in Software Configuration section). – Felipe Mariano Oct 23 '17 at 18:50
  • What is the default setting for `server.tomcat.threads.max`? – shashwat Feb 09 '22 at 13:14
  • I usted this property (server.tomcat.threads.max) and also tried server.tomcat.max-threads but my embebbed tomcat is still able to process 200 threads max at a time, were you able to solve it ? thanks – Diego Ramos Mar 20 '22 at 01:08
12

If you are developing web applications with Spring Boot (I mean that you have included the dependency of spring-boot-starter-web into your pom file), Spring will automatically embed web container (Tomcat by default) and it can handle requests simultaneously just like common web containers.

And you can also change the default web container from Tomcat to Undertow or Jetty by modifying the dependencies.

As @Felipe Mariano said, you can restrict the maximum amount of worker threads for different web container in your configuration file below.
(1) For Tomcat: server.tomcat.max-threads
(2) For Undertow: server.undertow.worker-threads
(3) For Jetty: server.jetty.acceptors

LHCHIN
  • 3,679
  • 2
  • 16
  • 34
0

There is a servlet container created for each request if I am right and is processed by each separate new spawned Thread so, technically each request is processed in paralleled. You need to configure the max threads in app properties file . Based upon your configurations thread pool will be taken care by Spring Framework . Thanks

Maiden
  • 207
  • 3
  • 4