1

A Tomcat 8 server has been serving as a reverse proxy behind apache httpd on CentOS 7. Tomcat has had three services running, each with an ajp connection to apache. But when I add a fourth service with its own fourth ajp connection, apache stops responding altogether. How can I assign a fourth ajp connection?

/opt/tomcat/conf/server.xml is:

<?xml version='1.0' encoding='utf-8'?>

<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.coyote.ajp.AjpProtocol" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">
      <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" />
      <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" connectionTimeout="10000" keepAliveTimeout="10000" />
      <Engine name="Catalina" defaultHost="localhost">
          <Realm className="org.apache.catalina.realm.LockOutRealm">
               <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
          </Realm>
          <Host name="localhost"  appBase="webapps" unpackWARs="true" autoDeploy="true">
               <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                   prefix="localhost_access_log" suffix=".txt"
                   pattern="%h %l %u %t &quot;%r&quot; %s %b" />
          </Host>
      </Engine>
  </Service>

  <Service name="Upload">
      <Connector port="8081" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8444" />
      <Connector port="8010" protocol="AJP/1.3" redirectPort="8444" connectionTimeout="10000" keepAliveTimeout="10000" />
      <Engine name="Catalina" defaultHost="localhost">
          <Realm className="org.apache.catalina.realm.LockOutRealm">
               <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
          </Realm>
          <Host name="localhost"  appBase="webapps_upload" unpackWARs="true" autoDeploy="true">
               <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                    prefix="localhost_access_log" suffix=".txt"
                    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
          </Host>
      </Engine>
  </Service>

  <Service name="Public">
      <Connector port="8082" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8445" />
      <Connector port="8011" protocol="AJP/1.3" redirectPort="8445" connectionTimeout="10000" keepAliveTimeout="10000" />
      <Engine name="Catalina" defaultHost="localhost">
           <Realm className="org.apache.catalina.realm.LockOutRealm">
               <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
           </Realm>
           <Host name="localhost"  appBase="webapps_public" unpackWARs="true" autoDeploy="true">
               <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                    prefix="localhost_access_log" suffix=".txt"
                    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
            </Host>
      </Engine>
  </Service>

  <Service name="domain3">
      <Connector port="8083" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8446" />
      <Connector port="8012" protocol="AJP/1.3" redirectPort="8446" connectionTimeout="10000" keepAliveTimeout="10000" />

      <Engine name="Catalina" defaultHost="localhost">
           <Realm className="org.apache.catalina.realm.LockOutRealm">
                <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
           </Realm>
           <Host name="localhost"  appBase="webapps_domain3" unpackWARs="true" autoDeploy="true">
               <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                    prefix="localhost_access_log" suffix=".txt"
                    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
           </Host>
       </Engine>
  </Service>

  <Service name="domain4">
       <Connector port="8084" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8447" />
       <Connector port="8013" protocol="AJP/1.3" redirectPort="8447" connectionTimeout="10000" keepAliveTimeout="10000" />

       <Engine name="Catalina" defaultHost="localhost">
           <Realm className="org.apache.catalina.realm.LockOutRealm">
                <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
           </Realm>
           <Host name="localhost"  appBase="webapps_domain4" unpackWARs="true" autoDeploy="true">
                <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                    prefix="localhost_access_log" suffix=".txt"
                    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
           </Host>
       </Engine>
  </Service>

</Server>

I tried adding the following inside the Public service above instead of adding a fourth service, but I am getting the same error:

            <!-- The following host is from: https://stackoverflow.com/questions/10704833/tomcat-and-multiple-domains-applications
                 <Host name="domain4.com"  appBase="webapps_domain4" unpackWARs="true" autoDeploy="true">
                      <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                           prefix="domain4_access_log." suffix=".txt"
                           pattern="%h %l %u %t &quot;%r&quot; %s %b" />
                 </Host>
            -->

The Apache logs are:

[Fri Oct 30 18:14:05.197127 2015] [proxy:error] [pid 25060] (111)Connection refused: AH00957: AJP: attempt to connect to SERVER.ip.addr:8011 (SERVER.ip.addr) fail$
[Fri Oct 30 18:14:05.197220 2015] [proxy:error] [pid 25060] AH00959: ap_proxy_connect_backend disabling worker for (SERVER.ip.addr) for 60s
[Fri Oct 30 18:14:05.197238 2015] [proxy_ajp:error] [pid 25060] [client remote.CLIENT.ip:54548] AH00896: failed to make connection to backend: SERVER.ip.addr

I read this other posting and this second other posting, but they did not answer my question.

Also, the httpd.conf has not changed, so I am omitting it for now to stay on topic as a java tomcat question.

Community
  • 1
  • 1
CodeMed
  • 9,527
  • 70
  • 212
  • 364
  • Terminology: apache is the reverse proxy. It accepts connections (contrary to a forward proxy that makes them) on behalf of the clients, which in this case are application servers. That out of the way: Is tomcat complaining when it stops responding and if yes, what's the error? That would be far more useful to know than that apache can't connect – zapl Oct 31 '15 at 00:03
  • @zapl which apache log file should i review to get the information you are requesting? – CodeMed Oct 31 '15 at 00:15
  • the tomcat log file. Not sure where you have it: https://wiki.apache.org/tomcat/FAQ/Logging maybe? – zapl Oct 31 '15 at 00:20
  • @zapl There are many tomcat log files including manager, localhost, localhost_access_log, host-manager, catalina.out, catalina.log – CodeMed Oct 31 '15 at 00:26
  • Probably catalina.log but could be anywhere. Is there any relevant error in any of them? And assuming `AJP: attempt to connect to SERVER.ip.addr:8011` is correct, httpd tries to connect to the AJP port of `Service name="Public"`, not of domain4. So maybe you use some wrong url / did not do some required change in httpd.conf? Don't you have to add something like `ProxyPass /apps/domain4 ajp://server.ip:8013` to httpd.conf? – zapl Oct 31 '15 at 00:40
  • @zapl I will make a thorough update soon as I attempt to reproduce the problem now. – CodeMed Oct 31 '15 at 00:43
  • @zapl all the logs in /opt/tomcat/logs are from 5 minutes before the apache logs. The tomcat logs have to do with shutting down and restarting tomcat. Then the apache logs are 5 minutes later with the request. Therefore, the request to apache did not reach tomcat. – CodeMed Oct 31 '15 at 00:56
  • @zapl catalina.out is giving a permgen error when i try to start tomcat. the error is related to multithreading. If you are around, can you ping me? I will take a different approach if you are engaged in this than I would if I have to re-document something for a general audience. – CodeMed Oct 31 '15 at 01:17

0 Answers0