I develop a web portal to give the user a way to access the information of an existing system. It is build on Java, running on Tomcat, Linux and MySQL. the user rarely access the system. Probably once every week to turn on and turn off some stuff.
The strange thing is, sometime to time, our system will encounter database connection issue and if the user refresh the browser, it will work again. Below is the stack trace when the connection issue occur:
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 648,770,075 milliseconds ago. The last packet sent successfully to the server was 648,770,076 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3988)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2598)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2825)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2156)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2323)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76)
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:646)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589)
... 96 more
Caused by: java.net.SocketException: Connection timed out
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
Note, i did not change the MySQL wait_timeout, which means it is by default wait_timeout=28800
What i dont understand is the error prompted that the connection was dorment around 7 days? the error message: "The last packet successfully received from the server was 648,770,075 milliseconds ago". But i thought i set my C3P0 connection correctly by testing the connection from time to time, set the maxConnectionAge, and test connection every check out by setting testConnectionOnCheckout=true. Below is the configuration:
<bean id="dataSource"
class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<property name="driverClass" value="com.mysql.jdbc.Driver" />
<property name="jdbcUrl" value="jdbc:mysql://localhost:3306/SocialAction?useEncoding=true&characterEncoding=UTF-8&characterResultSets=utf8&connectionCollation=utf8mb4_unicode_ci" />
<property name="user" value="username" />
<property name="password" value="password" />
<property name="minPoolSize" value="5" />
<property name="maxPoolSize" value="400" />
<property name="maxIdleTime" value="120" />
<property name="maxConnectionAge" value="7200" />
<property name="acquireRetryAttempts" value="5" />
<property name="maxStatements" value="100" />
<property name="idleConnectionTestPeriod" value="60" />
<property name="testConnectionOnCheckout" value="true" />
<property name="preferredTestQuery" value="SELECT 1" />
<property name="unreturnedConnectionTimeout" value="240" />
<property name="debugUnreturnedConnectionStackTraces" value="false" />
</bean>
The stack trace above suggest to use AutoReconnect=true , but because the community is suggesting differently "Why does autoReconnect=true not seem to work?". i am not going down to that path
Hence, the million dollar question is, why this happen while i have testConnectionOnCheckout=true and idleConnectionTestPeriod=true ? I recently added maxIdleTime=120, but it just does not solve this issue. Do you see anything wrong in my C3P0 configuration?