29

I would like to understand Tomcat's thread model for BIO and NIO connectors. I am referencing the official Tomcat 7 documentaion for connectors which can be found here. Based on it, this is what I have in doubt:

  • acceptorThread(s) : This is a single or at the most 2 threads (as mentioned in the doc) which is responsible only for accepting in-coming connections. This can be configured using acceptorThreadCount, and it's suggested that more than two can be used for a multi-cpu machine -
    • why is this ?
    • Does this imply that the number of simultaneous open connections scales with the number of cpus versus the number of open file descriptors allowed on the server system ?
  • maxConnections(s) :
    • What is the relation between this setting and acceptCount and the number of open file descriptors on the system.
    • Why is the default value for this so much higher for the NIO connector ( 10000 ) than for the BIO ( = maxThreads ) ?
  • acceptCount : This is the queue for requests when all request processing threads are busy.
    • When requests are put into this queue, is a file descriptor assigned to it as well ? Or is it only when a request is being actively processed, does it consume a file descriptor ?
  • request processing threads : The number of threads in this pool is configured by the maxThreads and minSpareThreads attributes.
    • What is the relation between this thread pool and the acceptorThreads ? Does the acceptor thread(s) spawn the threads in this pool ?
    • As I understand, the NIO model is more efficient with the request processing threads than the BIO model. How does it achieve this efficiency ?
    • As I've read in various sources, threads in the NIO model follow the thread per request paradigm vs the thread per connection paradigm of the BIO model. Also, in the NIO connector model, the actual request processing is delegated to a different, application monitored thread while the server's request processing thread is returned to the thread pool- free to accept more connections. So, does this imply that the benefit of the NIO model will only be apparent if the connections to the server are of the HTTP Keep-Alive nature or if the application is using Servlet 3.0's asynchronous processing feature ?
  • Servlet 3.0 :
    • When using Servlet 3.0, what should be the size of the application servlet thread pool ( relative to the connector thread pool size ) to achieve optimum efficiency ?
    • When using the BIO model and this together, will there be any difference as to how requests are processed ( given that the connector threads will still be using the thread per connection model ) ?

Please note: All discussion with respect to tomcat 7.

mahasamatman
  • 603
  • 1
  • 7
  • 10

1 Answers1

15

  • acceptorThread(s) : This is a single or at the most 2 threads (as mentioned in the doc) which is responsible only for accepting in-coming connections. This can be configured using acceptorThreadCount, and it's suggested that more than two can be used for a multi-cpu machine -

    why is this ?

Accepting connections is a very low cost operation so it just doesn't make sense to dedicate more than one thread to this task.

Does this imply that the number of simultaneous open connections 
scales with the number of cpus versus the number of open file descriptors 
allowed on the server system ?

No it doesn't because it's a very low cost operation in terms of CPU. As for file descriptors, each accepted connection is going to use a file descriptor, so the maxim number of the connections the server can accept is limited by the number of available file descriptors.

  • maxConnections(s) :

    What is the relation between this setting and acceptCount and the number of open file descriptors on the system.

maxConnections cannot be higher the number of open file descriptors on the system. Keep in mind that other processes use file descriptors as well, so may want to be conservative with maxConnections in regards to the available file descriptors, let's say maxConnections < file descriptors / 2. 

Why is the default value for this so much higher for the NIO connector 
( 10000 ) than for the BIO ( = maxThreads ) ?

This is because in NIO a single handles all IO, and in BIO the server need to create/use a separate thread-per connection.

  • acceptCount : This is the queue for requests when all request processing threads are busy.

    When requests are put into this queue, is a file descriptor assigned to it as well ?

Yes, that's correct the connection request is accepted but the server not yet ready to being serving requests, so the connection is placed in the queue. This is done to prevent TCP/stack timing out connection requests which may look like a server down from the client point of view. In other words, server says 'I'm here and will process your request once I have resources to do it'.

Or is it only when a request is being actively processed, does it consume a file descriptor ?

Nope.

Hope this helps.

Regards,

Slava Imeshev

Slava Imeshev
  • 1,360
  • 10
  • 14
  • 1
    Thanks a lot for your answer. Can you speculate on one more question as well? To build a really scalable server (handling about 10000 requests concurrently, as per current requirements), is NIO the only way to go? I ask this because, as per many literature, the BIO connector is the more stable one (though I'm not quite sure what is implied by 'stable' here). Or should I switch away from TC entirely and look at other event based alternatives such as jetty or jboss? – mahasamatman Nov 11 '14 at 14:20
  • 1
    I'd say yes, NIO is the way to go. The thing is, you will be able to handle the number of connections, your application's a ability to process those requests in parallel could be a different story. You may need to run multiple servers in the end, so you have to consider historical scalability since the beginning and design the system accordingly. By the way, Tomcat has NIO connector: https://tomcat.apache.org/tomcat-7.0-doc/config/http.html#NIO_specific_configuration – Slava Imeshev Nov 13 '14 at 02:05