Tomcat8 configuration "maxThreads" does not work

Question

You can find the maxThreads limitation is 350 in below server.xml.

Then, I use below command to check current java pid

$ps uax | grep tomcat
ruyi     25802  0.0  0.0  10436   420 ?        Ss   Oct14   0:00 jsvc.exec -jvm server -java-home /usr/java/jdk1.8.0_51 -user ruyi -pidfile /data/tomcat/logs/catalina-daemon.pid -wait 10 -outfile /data/tomcat/logs/catalina-daemon.out -errfile &1 -classpath /data/tomcat/bin/bootstrap.jar:/data/tomcat/bin/commons-daemon.jar:/data/tomcat/bin/tomcat-juli.jar -Djava.util.logging.config.file=/data/tomcat/conf/logging.properties -server -Djava.net.preferIPv4Stack=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dfile.encoding=UTF8 -Xmx4096m -XX:PermSize=256m -XX:MaxPermSize=512m -Djava.endorsed.dirs= -Dcatalina.base=/data/tomcat -Dcatalina.home=/data/tomcat -Djava.io.tmpdir=/data/tomcat/temp org.apache.catalina.startup.Bootstrap
ruyi     25803  0.6 18.5 9458412 3015712 ?     Sl   Oct14  33:24 jsvc.exec -jvm server -java-home /usr/java/jdk1.8.0_51 -user ruyi -pidfile /data/tomcat/logs/catalina-daemon.pid -wait 10 -outfile /data/tomcat/logs/catalina-daemon.out -errfile &1 -classpath /data/tomcat/bin/bootstrap.jar:/data/tomcat/bin/commons-daemon.jar:/data/tomcat/bin/tomcat-juli.jar -Djava.util.logging.config.file=/data/tomcat/conf/logging.properties -server -Djava.net.preferIPv4Stack=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dfile.encoding=UTF8 -Xmx4096m -XX:PermSize=256m -XX:MaxPermSize=512m -Djava.endorsed.dirs= -Dcatalina.base=/data/tomcat -Dcatalina.home=/data/tomcat -Djava.io.tmpdir=/data/tomcat/temp org.apache.catalina.startup.Bootstrap
ruyi     30257  0.0  0.0 103304   892 pts/1    S+   11:29   0:00 grep tomcat

then, I use command watch -n 5 -d "ps -eL <java_pid> | wc -l" to check current thread number is 1429, which is bigger than the maxThreads limitation 350.

So, my question is why configuration maxThreads doesn't work, and how to check the configuration of a running tomcat (to see whether my configuration is set correctly).

Here is the server.xml

<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
    -->

<Connector executor="tomcatThreadPool" port="8080" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="350" maxHttpHeaderSize="8192" emptySessionPath="true"
   connectionTimeout="20000" enableLookups="false" maxPostSize="0"
                  redirectPort="8443"  URIEncoding="UTF-8" />
<!--
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" /> -->
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
         This connector uses the NIO implementation that requires the JSSE
         style configuration. When using the APR/native implementation, the
         OpenSSL style configuration is required as described in the APR/native
         documentation -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
               maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

UPDATE

Thanks @adamM, I just use jstack to check current running threads, and find most of them are below threads, any ideas?

 "pool-366-thread-1" #1437 prio=5 os_prio=0 tid=0x00007f55ccbb7000 nid=0x2fa9 waiting on condition [0x00007f55b9ffa000]
    java.lang.Thread.State: WAITING (parking)
         at sun.misc.Unsafe.park(Native Method)
         - parking to wait for  <0x00000006f482a018> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
         at java.lang.Thread.run(Thread.java:745)

score 2 · Accepted Answer · edited May 23 '17 at 10:27

2

maxThreads configures the number of servlet request threads (threads that do work to respond to request). If you set it to 350, that should be the limit of those threads, but there may be more threads in the JVM that are doing other things. You can use a profiler to look at the name of the threads in the JVM. The thread's name will give you an idea of what is causing those extra threads.

IF you don't want to use a profiler, you can do this in code by getting all the current threads and printing their names. Take a look at this post:

Get a List of all Threads currently running in Java

edited May 23 '17 at 10:27

Community

1
1

answered Oct 18 '16 at 03:46

adamM

1,116
10
29

what happens when you use jconsole (a little app that comes with teh jdk), it allows you to see the names of all running threads. I think your orginal number of 1429 was not accurate. Also, what is tomcat doing? Are you running an appp, or is it just an empty tomcat instance running? – adamM Oct 18 '16 at 04:11
thanks, you have answered my original question, I'll open a new thread to discuss this question. – Yuwen Yan Oct 18 '16 at 08:49

score 0 · Answer 2 · answered May 11 '18 at 09:01

Threads in JVM would be doing lots of things. Rather than using ps -eL <java_pid> | wc -l,

Get the different running threads from the list using ps -eL <java_pid> and try like following examples,

ps -eL <java_pid> | grep "/usr/lib/jvm/java-8-oracle/bin/java" | wc -l
ps -eL <java_pid> | grep "/usr/sbin/mysqld" | wc -l
ps -eL <java_pid> | grep "/usr/sbin/apache2" | wc -l

Tomcat8 configuration "maxThreads" does not work

UPDATE

2 Answers2