Tomcat 7 memory leak with spring-mvc + jpa/hibernate + mysql

Question

I am building a relatively simple web app using

spring-mvc 3.2.0
spring-jpa 1.2.0
jpa 2.0
hibernate 3.6
MySql 5.6 (using mysql jdbc driver 5.1.28)

I dont initialize a connection myself, only use autowired repository to read or write entities.

Problem: every time when:

The app is stopped (redeployed, etc)
A user session expires

I am getting a memory leak on tomcat server and the following catalina log:

Feb 02, 2014 1:31:12 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc 
SEVERE: The web application [/test] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Feb 02, 2014 1:31:12 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads 
SEVERE: The web application [/test] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
Feb 02, 2014 1:31:12 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks 
SEVERE: The web application [/test] created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [java.util.HashMap] (value [{public abstract java.util.List org.springframework.data.jpa.repository.JpaRepository.findAll()=java.lang.Object@632cc9ed, public abstract java.util.List data.repository.QuestionRepository.findAllNonDisabled()=java.lang.Object@632cc9ed, public abstract java.util.List data.repository.AnswerRepository.findAllAnswersForUserAndGameOrderBySortIndex(int,int)=java.lang.Object@632cc9ed, public abstract java.lang.Object org.springframework.data.repository.CrudRepository.save(java.lang.Object)=java.lang.Object@632cc9ed, public abstract java.lang.Object org.springframework.data.repository.CrudRepository.findOne(java.io.Serializable)=java.lang.Object@632cc9ed}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Feb 02, 2014 1:31:12 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
.......
SEVERE: The web application [/test] created a ThreadLocal with stop
INFO: The stop() method was called on component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/test]] after stop() had already been called. The second call will be ignored.
Feb 02, 2014 1:42:26 PM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/test]

What this log says : any query ever executed against the database (including default ones like findOne and findAll) are being held in memory and cannot be unloaded.

I've read about some workarounds (like moving the mysql driver into the lib folder of tomcat ), but i dont believe that this is a general issue and I think it is cause by my app's configuration in one way or another.

Here is my configuration:

Servlet config web.xml

<web-app version="2.4"
xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee 
http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

<display-name>GPMM 2014 Feedback</display-name>

<servlet>
    <servlet-name>mvc-dispatcher</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
    <servlet-name>mvc-dispatcher</servlet-name>
    <url-pattern>/</url-pattern>
</servlet-mapping>

<!-- Spring Security -->
<context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>/WEB-INF/security.xml</param-value>
</context-param>

<servlet-mapping>
    <servlet-name>default</servlet-name>
    <url-pattern>/resources/*</url-pattern>
</servlet-mapping>

<listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>

<filter>
    <filter-name>springSecurityFilterChain</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
</filter>

<filter-mapping>
    <filter-name>springSecurityFilterChain</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Spring Security -->

<!-- File upload -->
<filter>
    <filter-name>multipartFilter</filter-name>
    <filter-class>org.springframework.web.multipart.support.MultipartFilter</filter-class>
</filter>

<filter-mapping>
    <filter-name>multipartFilter</filter-name>
    <url-pattern>/admin/**</url-pattern>
</filter-mapping>
<!-- File upload -->

Spring mvc bean config:

<beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:context="http://www.springframework.org/schema/context"
   xmlns:mvc="http://www.springframework.org/schema/mvc"
   xmlns:tx="http://www.springframework.org/schema/tx"
   xmlns:jpa="http://www.springframework.org/schema/data/jpa" xmlns:p="http://www.springframework.org/schema/p"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/data/jpa http://www.springframework.org/schema/data/jpa/spring-jpa.xsd">

<context:component-scan base-package="com.springapp.mvc.controller.admin"/>
<context:component-scan base-package="com.springapp.mvc.controller.user"/>

<bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    <property name="prefix" value="/WEB-INF/pages/"/>
    <property name="suffix" value=".jsp"/>
</bean>

<bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
    <property name="maxUploadSize" value="2000000"/> <!-- File size in bytes. -->
</bean>

<!-- Data access management -->
<jpa:repositories base-package="data.repository"/>

<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalEntityManagerFactoryBean">
    <property name="persistenceUnitName" value="defaultPersistenceUnit"/>
</bean>

<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
    <property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>

<tx:annotation-driven transaction-manager="transactionManager"/>
<!-- Data access maanagement -->

Security config:

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:beans="http://www.springframework.org/schema/beans"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
   http://www.springframework.org/schema/security
   http://www.springframework.org/schema/security/spring-security.xsd">

<http auto-config="true" authentication-manager-ref="adminauth">
    <intercept-url pattern="/admin/**" access="ROLE_ADMIN" />
    <intercept-url pattern="/updatedb/**" access="ROLE_DBADMIN" />
    <!--<intercept-url pattern="/game/**" access="ROLE_USER" />-->
    <form-login login-page="/adminlogin" authentication-failure-url="/accessdenied" />
    <logout logout-success-url="/admin" logout-url="/j_spring_security_logout" />
</http>

<authentication-manager id="adminauth">
    <authentication-provider>
        <password-encoder hash="sha-256"/>
        <user-service>
            <user name="admin" password="xxx" authorities="ROLE_ADMIN" />
            <user name="dbadmin" password="xxx" authorities="ROLE_DBADMIN" />
            <user name="user" password="xxx" authorities="ROLE_USER" />
        </user-service>
    </authentication-provider>
</authentication-manager>

</beans:beans>

And JPA/Hibernate config:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="defaultPersistenceUnit" transaction-type="RESOURCE_LOCAL">
    <provider>org.hibernate.ejb.HibernatePersistence</provider>
    <class>data.model.TestEntity</class>
    <properties>
        <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect" />
        <property name="hibernate.connection.url" value="jdbc:mysql://localhost:3306/gpm" />
        <property name="hibernate.connection.driver_class" value="com.mysql.jdbc.Driver" />
        <property name="hibernate.connection.username" value="user" />
        <property name="hibernate.connection.password" value="password" />
        <property name="hibernate.hbm2ddl.auto" value="validate" />
    </properties>
</persistence-unit>

And this is a typical code sample:

@Controller
public class TestController
{
static final Logger logger = LogManager.getLogger(TestController.class);

@Autowired
private TestRepository testRepository;


@RequestMapping(value = "/admin/tests", method = RequestMethod.GET)
public String listTests(ModelMap model)
{
    TestEntity gamegroup = new TestEntity();

    model.addAttribute("test", test);
    model.addAttribute("tests", testRepository.findAll());

    return "admin/tests";
}

@RequestMapping(value = "/admin/tests/update", method = RequestMethod.POST)
public String updateTest(@ModelAttribute("test") TestEntity test, BindingResult result)
{
    testRepository.save(test);

    return "redirect:/admin/tests";
}
}

Please point me out what is could be wrong with my configuration or the way i access mysql.

The problem is the MySQL thread. See [here](http://stackoverflow.com/questions/11872316/tomcat-guice-jdbc-memory-leak). — Sotirios Delimanolis, Feb 02 '14 at 15:05
Thanks, i've seen your post and actually implemenented the listener, testing it now, but i have 4 different leaks (jdbc, transaction, connection and one i dont understand yet). Sometimes i feel i offended a god in my last life — BanditoBunny, Feb 02 '14 at 15:21
It looks like the solution solves jdbc and connection problem, unfortunately not the transaction one :( — BanditoBunny, Feb 02 '14 at 15:43

score 2 · Answer 1 · edited May 23 '17 at 11:43

ThreadLocal Issue I'm tracking down a similar problem and one thing that you should do is add @Transactional to any request mappings that touch your database. This makes these interactions transactional so they can be rolled back if something goes wrong. See This blog post

WebClassLoader Issue You're loading your driver inside your servlet instead of having it managed by your container(Tomcat). You can unload it yourself in a listener on shutdown, or you can let Tomcat manage it. There are a LOT of debates about which way to go. I like managing it myself to make the management for the admin easier. However it isn't as flexible for admins who know what they're doing. See this stack overflow

This blog post also helped: https://blog.bosch-si.com/developer/how-to-analyze-leaky-webapps/ — Pedro Madrid, Apr 10 '18 at 18:50

Tomcat 7 memory leak with spring-mvc + jpa/hibernate + mysql

1 Answers1