I am having intermittent issue when executing a simple COUNT(*) stored procedure from Java. I am getting below java.sql.SQLException with incompatible types. See the example stored procedure which is not doing much except COUNT(*).
The weird thing is, below exception doesn't happen all the time but happens only after some time. Please note that COUNT on below procedure will always return very small values like 40 or 50. It never in fact goes to 3 digit number so it should be OK as integer.
What I am not able to understand is, if it is actual data type mismatch than why does it work sometimes and suddenly stopped working without any data update happen on the database.
So everything working fine right now and suddenly everything stopped working after few mins or after few request. It is very random and it suddenly starts throwing below error without any change on data in database. It starts working again if I restart the server or redeploy the same code without changing any code or data within database. It is deploying this within embedded tomcat environment.
Please check below JDBC connection code plus configurations that I am doing while connecting to database. Also please note that variable which holds value for that count column is defined as Integer and not as Long at this time. I can probably fix this by simply changing variable type here but not sure I want to change that without understanding why this works fine sometimes and starts breaking suddenly.
There are many data points which throws similar type of type failure errors so changing it to all of them might be difficult. BTW it starts working again if I restart the server or redeploy the same code without changing any code or data within database.
Please find below exceptions that it throws sometimes but not all the times.
2018-10-11T19:50:45.175782Z app[web.1]: {"remoteAddress": "172.28.","serverName": "test","remoteUser": "-","timeLocal": "11/Oct/2018:19:50:45 +0000","request": "GET /v1/test?input=ABCD&_type=json&limit=200 HTTP/1.1","status": "500","bodyBytesSent": 127,"httpReferer": "-","httpUserAgent": "-","requestTime": "0.013","upstreamCacheStatus": "MISS","httpXForwardedFor": "172.28.9.10","suuid": "-","puuid": "-","env": "PPE"}
2018-10-11T19:50:45.181112Z app[web.1]: 2018-10-11T19:50:45,178 test ERROR [http-nio-8088-exec-2,,,] com.aaa.us.test.ws.dao.DataDao: [proc_test] Failed to fetch data for input - ABCD
2018-10-11T19:50:45.181152Z app[web.1]: java.sql.SQLException: Cannot set count: incompatible types, cannot convert java.lang.Long to int Query: CALL proc_test(?); Parameters: [ABCD]
2018-10-11T19:50:45.181162Z app[web.1]: at org.apache.commons.dbutils.AbstractQueryRunner.rethrow(AbstractQueryRunner.java:527) ~[commons-dbutils-1.7.jar!/:1.7]
2018-10-11T19:50:45.181186Z app[web.1]: at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:391) ~[commons-dbutils-1.7.jar!/:1.7]
2018-10-11T19:50:45.1812Z app[web.1]: at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:329) ~[commons-dbutils-1.7.jar!/:1.7]
2018-10-11T19:50:45.181213Z app[web.1]: at com.aaa.us.test.ws.dao.getData(DataDao.java:200) [classes!/:1.0.1]
2018-10-11T19:50:45.181228Z app[web.1]: at com.aaa.us.test.ws.DataImpl.getData(DataImpl.java:98) [classes!/:1.0.1]
2018-10-11T19:50:45.181241Z app[web.1]: at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) ~[?:?]
2018-10-11T19:50:45.181252Z app[web.1]: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181-heroku]
2018-10-11T19:50:45.181265Z app[web.1]: at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181-heroku]
2018-10-11T19:50:45.181275Z app[web.1]: at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179) [cxf-core-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181286Z app[web.1]: at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) [cxf-core-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181294Z app[web.1]: at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:192) [cxf-rt-frontend-jaxrs-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181306Z app[web.1]: at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:103) [cxf-rt-frontend-jaxrs-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181316Z app[web.1]: at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) [cxf-core-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181326Z app[web.1]: at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) [cxf-core-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181352Z app[web.1]: at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) [cxf-core-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181364Z app[web.1]: at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) [cxf-core-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181376Z app[web.1]: at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181385Z app[web.1]: at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181396Z app[web.1]: at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181406Z app[web.1]: at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181414Z app[web.1]: at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:191) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181423Z app[web.1]: at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:301) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181432Z app[web.1]: at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:225) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.182000+00:00 system[router]: at=info method=GET path=/v1/data?input=ABCD&_type=json&limit=10 host=test fwd="172.28.9.10" zone=va container=web.1 connect=0ms service=13ms status=500 bytes=475 request_id=e1aa4b25-f8ab-48c4-a89f-09fe9867d273
2018-10-11T19:50:45.18144Z app[web.1]: at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181449Z app[web.1]: at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:276) [cxf-rt-transports-http-3.2.0.jar!/:3.2.0]
2018-10-11T19:50:45.181458Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181467Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181476Z app[web.1]: at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181486Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181497Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181508Z app[web.1]: at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) [spring-boot-actuator-1.5.9.RELEASE.jar!/:1.5.9.RELEASE]
2018-10-11T19:50:45.181516Z app[web.1]: at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181524Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181533Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181544Z app[web.1]: at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181553Z app[web.1]: at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181565Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181577Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181587Z app[web.1]: at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181601Z app[web.1]: at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181612Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181622Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181633Z app[web.1]: at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181644Z app[web.1]: at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181656Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181666Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181677Z app[web.1]: at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181688Z app[web.1]: at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181697Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181708Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
2018-10-11T19:50:45.181721Z app[web.1]: at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106) [spring-boot-actuator-1.5.9.RELEASE.jar!/:1.5.9.RELEASE]
2018-10-11T19:50:45.181731Z app[web.1]: at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
2018-10-11T19:50:45.181742Z app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar!/:8.5.23]
Please note that we are using apache dbutil commons library to connect to DB here.
import org.apache.commons.dbutils.QueryRunner;
import org.apache.commons.dbutils.ResultSetHandler;
import org.apache.commons.dbutils.handlers.BeanHandler;
import org.apache.commons.dbutils.handlers.BeanListHandler;
try {
QueryRunner run = new QueryRunner(readDataSource);
ResultSetHandler<Dto> h1 = new BeanHandler(Dto.class);
Dto data = (run.query("CALL proc_WS_GetAlpsHoldings_Metadata(?);", h1, input));
response.setMeta(extendedMetaModel(intPage, intLimit, holdings.size(), data.getCount(), input, null, data.getDate()));
} catch (SQLException ex) {
log.error("[proc_test] Failed to fetch data for input - " + input, ex);
throw new GenericException().errorCode(500).errorMessage("Failed to fetch data for input - " + input);
}
Dto class where I have defined count value as int is as below.
public class Dto {
private String date;
private String inceptionDate;
private int count;
public String getAsOfDate() {
return asOfDate;
}
public void setDate(String date) {
this.date = date;
}
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
}
Please find below JDBC connection pool properties that we set.
datasource:
dbClientvendordata:
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://testmysqlread.something.com:3306/db_testdb?useOldAliasMetadataBehavior=true
username: "test"
password: "test"
auth: Container
maxActive: 10
maxIdle: 1
maxWait: 120000
testWhileIdle: true
timeBetweenEvictionRunsMillis: 10000
minEvictableIdleTimeMillis: 60000
removeAbandoned: true
removeAbandonedTimeout: 300
noAccessToProcedureBodies: true
Please find below simple stored procedure code.
CREATE PROCEDURE `proc_test`(
input VARCHAR (10)
)
BEGIN
SELECT
`column1` AS `Ticker`,
DATE_FORMAT(`date`, '%Y-%m-%d') AS `date`,
COUNT(*) AS `count`
FROM
`db_testdb`.`table1`
WHERE
`column1` = input
GROUP BY
`column1`;
END$$
DELIMITER ;
