0

I'm facing a pretty annoying problem in my JBoss application, the system is crashing at some times of the day and I can not figure out why.

Monitoring the application by VisualVM I can see that some threads are waiting for something that I also can not figure out what.

Analyzing the Thread Dump the stuck threads receive the following log:

"http-/0.0.0.0:8080-459" - Thread t@2309
   java.lang.Thread.State: WAITING
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <105c9d1d> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:131)
        at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:281)
        at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:62)
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:176)
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:169)
        at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:100)
        at org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:212)
        at org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:199)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:456)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
        at adapter.proxy.controller.ProxyServlet.service(ProxyServlet.java:291)
        at adapter.proxy.controller.Proxy.service(Proxy.java:46)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
        at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50)
        at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50)
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

I would like some hint or tool that I can use to find out what is making the application hang. If it is necessary for me to provide any more information, just ask.

The image below shows some Threads of the system, the moment where several are with the status 'park' (orange) is when it hangs.

Threads

Gian Honório
  • 53
  • 1
  • 11
  • 1
    This looks like a thread waiting in a thread pool. This is expected and correct behavior. – korolar Apr 04 '18 at 14:38
  • Expected and correct sure, but if you've got threads waiting to get a connection it can be a symptom of connection leaks, deadlocks or not having enough resources for the work you're doing. You don't want to wait when getting a connection from the pool after all. It's a connection pool that's blocking, there's nothing related to threadpools here. – Kayaman Apr 04 '18 at 14:50
  • Thanks for the answers. I found this behavior strange because it always grabs when the system hangs, but as you said it seems to be just a symptom of something else. It was mentioned that the problem may be in the connection pool, you could recommend me some method or tool that I can use to monitor it, and detect possible problems? Currently I am using JBoss EAP 6.3 and a MySQL database. – Gian Honório Apr 04 '18 at 15:14
  • Actually that's not a database connection pool. It's the HTTP connection pool for Apache's HTTP client. Search around and see if there are known problems with the pool running out of connections. – Kayaman Apr 04 '18 at 15:30

0 Answers0