6

Currently, if the database of our Spring app hosted on Tomcat is unavailable, context initialization fails and all requests return a 404.

What would be a good way to overcome this problem? Instead of the app being unavailable until the next Tomcat, I would like it to show an error message to the user while unavailable and automatically recover when the database is available (like it would if a database failure occurs while Tomcat is already running).

I can set all beans to lazy-init, but I'm not sure that's the best solution? Can't Tomcat retry the initialization every x seconds/requests and show a decent error page in the meantime? Any ideas on this?

An example of the errors thrown on startup when the DB is unavailable:

Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:529)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128)
at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.getTargetConnection(LazyConnectionDataSourceProxy.java:401)
at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:376)
at com.sun.proxy.$Proxy43.getMetaData(Unknown Source)
at com.googlecode.flyway.core.dbsupport.DbSupportFactory.getDatabaseProductName(DbSupportFactory.java:103)
... 67 more
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1319)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525)
... 72 more
Mar 28, 2013 11:19:47 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error listenerStart
Mar 28, 2013 11:19:47 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [] startup failed due to previous errors
MikeN
  • 887
  • 5
  • 18
  • maybe it would be wise to focus on making sure your db is available/reliable first ... but a custom 404 is pretty simple : http://stackoverflow.com/questions/1196569/custom-404-using-spring-dispatcherservlet – NimChimpsky Mar 28 '13 at 12:34
  • I can't expect from my customers to perform their Windows updates in exactly the right order, not in a server landscape with hundreds of servers, so sometimes the DB server may be down when the app server comes up. A custom 404 is not possible with the link you gave (since the webapp is not loaded), I could set up a custom 404 in Tomcat, but that would not solve the problem. What I want is for Spring/Tomcat to retry and try hard to get the app up, instead of failing after 1 try. – MikeN Mar 28 '13 at 15:49
  • Are you using connection pooling for data source? – abishkar bhattarai Apr 01 '13 at 06:17
  • Yes, but not sure how that would make a difference. – MikeN Apr 01 '13 at 09:20

1 Answers1

1

From the exception stack trace I see that you are using c3p0 connection pool. Look at http://www.mchange.com/projects/c3p0/#configuring_recovery where you can configure your pool to retry for connection.

But this would keep the http thread active and imagine if you have high load your system will go down.

Its best to show an error page to the user. In this case i would show a 503 error. You can do this by putting Apache web server in front of your tomcat where you host the 503 page. There could be other ways to achieve this also.

Since your webapp itself will not start you cannot serve the error page from your webapp unless you configure it to be lazy init.

You can also try auto refresh on the error page every X seconds for the url user was trying to visit.

Bhushan Bhangale
  • 10,921
  • 5
  • 43
  • 71
  • Thanks, but this is of course not the solution I'm looking for, since the web app will never start this way. – MikeN Apr 03 '13 at 09:10