Memoryleaks in tomcat at Stop

Question

we have some problem when we try to stop tomcat (or restart). The tomcat log is the following:

Mar 14, 2014 11:28:02 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/celcom/programas/jdk/jdk1.7.0_17/jre/lib/amd64/server:/home/celcom/programas/jdk/jdk1.7.0_17/jre/lib/amd64:/home/celcom/programas/jdk/jdk1.7.0_17/jre/../lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Mar 14, 2014 11:28:02 AM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'source' to 'org.eclipse.jst.jee.server:sms_services' did not find a matching property.
Mar 14, 2014 11:28:03 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-9393"]
Mar 14, 2014 11:28:03 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-9309"]
Mar 14, 2014 11:28:03 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 528 ms
Mar 14, 2014 11:28:03 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Mar 14, 2014 11:28:03 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.32
Mar 14, 2014 11:28:10 AM com.sun.faces.config.ConfigureListener contextInitialized
INFO: Initializing Mojarra 2.2.5 ( 20140108-1427 https://svn.java.net/svn/mojarra~svn/tags/2.2.5@12761) for context '/sms_services'
Mar 14, 2014 11:28:10 AM com.sun.faces.spi.InjectionProviderFactory createInstance
INFO: JSF1048: PostConstruct/PreDestroy annotations present.  ManagedBeans methods marked with these annotations will have said annotations processed.
Mar 14, 2014 11:28:10 AM com.sun.faces.application.ApplicationImpl getProjectStage
INFO: Unable to discern ProjectStage for value development.
Mar 14, 2014 11:28:10 AM com.sun.faces.config.processor.AbstractConfigProcessor getProjectStage
INFO: Unable to discern ProjectStage for value development.
Mar 14, 2014 11:28:11 AM org.primefaces.webapp.PostConstructApplicationEventListener processEvent
INFO: Running on PrimeFaces 4.0
Mar 14, 2014 11:28:15 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-9393"]
Mar 14, 2014 11:28:15 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9309"]
Mar 14, 2014 11:28:15 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 12343 ms
Mar 14, 2014 11:29:22 AM org.apache.catalina.core.StandardServer await
INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance.
Mar 14, 2014 11:29:22 AM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["http-bio-9393"]
Mar 14, 2014 11:29:22 AM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["ajp-bio-9309"]
Mar 14, 2014 11:29:22 AM org.apache.catalina.core.StandardService stopInternal
INFO: Stopping service Catalina
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Camel (camel-1) thread #0 - ManagementLoadTask] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Camel (camel-1) thread #1 - Batch Sender] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Camel (camel-1) thread #2 - Batch Sender] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Camel (camel-1) thread #3 - Batch Sender] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Camel (camel-1) thread #4 - Batch Sender] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sms_services] appears to have started a thread named [Camel (camel-1) thread #5 - Batch Sender] but has failed to stop it. This is very likely to create a memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sms_services] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@b4bdf15]) and a value of type [java.util.Hashtable] (value [{activemq.broker=localhost}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sms_services] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@b4bdf15]) and a value of type [java.util.Hashtable] (value [{activemq.broker=localhost}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sms_services] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@b4bdf15]) and a value of type [java.util.Hashtable] (value [{activemq.broker=localhost}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sms_services] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@b4bdf15]) and a value of type [java.util.Hashtable] (value [{activemq.broker=localhost}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sms_services] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@b4bdf15]) and a value of type [java.util.Hashtable] (value [{activemq.broker=localhost}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 14, 2014 11:29:22 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sms_services] created a ThreadLocal with key of type [org.apache.log4j.helpers.ThreadLocalMap] (value [org.apache.log4j.helpers.ThreadLocalMap@b4bdf15]) and a value of type [java.util.Hashtable] (value [{activemq.broker=localhost}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 14, 2014 11:29:22 AM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-bio-9393"]
Mar 14, 2014 11:29:22 AM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-bio-9309"]
Mar 14, 2014 11:29:22 AM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-bio-9393"]
Mar 14, 2014 11:29:22 AM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-bio-9309"]

The pom.xml of the xml is the following:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>sms_services</groupId>
    <artifactId>sms_services</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>
    <build>
        <sourceDirectory>src</sourceDirectory>
        <resources>
            <resource>
                <directory>src</directory>
                <excludes>
                    <exclude>**/*.java</exclude>
                </excludes>
            </resource>
            <resource>
                <directory>conf</directory>
                <excludes>
                    <exclude>**/*.java</exclude>
                </excludes>
            </resource>
        </resources>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.3</version>
                <configuration>
                    <warSourceDirectory>WebContent</warSourceDirectory>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>xom</groupId>
            <artifactId>xom</artifactId>
            <version>1.2.5</version>
        </dependency>
        <dependency>
            <groupId>org.apache.activemq</groupId>
            <artifactId>activemq-all</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.activemq</groupId>
            <artifactId>activemq-pool</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.activemq</groupId>
            <artifactId>activemq-camel</artifactId>
        </dependency>
        <dependency>
            <groupId>joda-time</groupId>
            <artifactId>joda-time</artifactId>
            <version>2.2</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate.javax.persistence</groupId>
            <artifactId>hibernate-jpa-2.0-api</artifactId>
            <version>1.0.1.Final</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>4.2.1.Final</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-entitymanager</artifactId>
            <version>4.2.1.Final</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.jackson</groupId>
            <artifactId>jackson-core-asl</artifactId>
            <version>1.9.12</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>apache-log4j-extras</artifactId>
            <version>1.1</version>
        </dependency>
        <dependency>
            <groupId>c3p0</groupId>
            <artifactId>c3p0</artifactId>
            <version>0.9.1.2</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-c3p0</artifactId>
            <version>4.2.2.Final</version>
        </dependency>
        <dependency>
            <groupId>org.jboss.resteasy</groupId>
            <artifactId>resteasy-jaxrs</artifactId>
            <version>3.0.5.Final</version>
        </dependency>
        <dependency>
            <groupId>axis</groupId>
            <artifactId>axis</artifactId>
            <version>1.4</version>
        </dependency>
        <dependency>
            <groupId>commons-httpclient</groupId>
            <artifactId>commons-httpclient</artifactId>
            <version>3.1</version>
        </dependency>
        <dependency>
            <groupId>javax.mail</groupId>
            <artifactId>javax.mail-api</artifactId>
            <version>1.5.1</version>
        </dependency>
        <dependency>
            <groupId>org.primefaces</groupId>
            <artifactId>primefaces</artifactId>
            <version>4.0</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.jackson</groupId>
            <artifactId>jackson-jaxrs</artifactId>
            <version>1.9.12</version>
        </dependency>
        <dependency>
            <groupId>org.quartz-scheduler</groupId>
            <artifactId>quartz</artifactId>
            <version>2.2.1</version>
        </dependency>
        <dependency>
            <groupId>net.oauth.core</groupId>
            <artifactId>oauth</artifactId>
            <version>20100527</version>
        </dependency>
        <dependency>
            <groupId>net.oauth.core</groupId>
            <artifactId>oauth-provider</artifactId>
            <version>20100527</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish</groupId>
            <artifactId>javax.faces</artifactId>
            <version>2.2.5</version>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.apache.activemq</groupId>
                <artifactId>activemq-all</artifactId>
                <version>5.8.0</version>
            </dependency>
            <dependency>
                <groupId>org.apache.camel</groupId>
                <artifactId>camel-core</artifactId>
                <version>2.10.3</version>
            </dependency>
            <dependency>
                <groupId>org.apache.activemq</groupId>
                <artifactId>activemq-pool</artifactId>
                <version>5.8.0</version>
            </dependency>
            <dependency>
                <groupId>org.apache.activemq</groupId>
                <artifactId>activemq-camel</artifactId>
                <version>5.8.0</version>
            </dependency>
        </dependencies>
    </dependencyManagement>
</project>

Well, another issue is that the Memory is growing up and we have to restart the servlet container (tomcat) like 2 or 3 days, so here is the question: How i can detect who is creating the memory leaks? i STFW but this problems it's has not clear solution (or maybe none applicated to camel+activemq+quartz+c3p0 that seams to be the problem.

If you required more info please ask and i will put it :)

Thx :D

UPDATE1:

C3P0 problems are solved in: Hibernate4 + c3p0 + Derby - Memory Leak on Tomcat stop or web application reload (*)

(*) In deed, killem all the conecctions cause that there isn't com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#X and reduce the stoping time considerably :) i hope that there is another workarounds to the rest of the API that i'm using :) — Enrique San Martín, Mar 14 '14 at 15:00

Enrique San Martín · Answer 1 · 2014-03-14T18:10:12.330

Ok, the problems of c3p0 are fixed in the update 1 of my question, about the problems of camel are fixed with the correct close of the camel context that i was using:

public static CamelContext CAMEL_CONTEXT = new DefaultCamelContext();

//some other code...

public static void stop() {
        try {
            if (CAMEL_CONTEXT != null) {
            CAMEL_CONTEXT.stop();
            }
        } catch (Exception e) {
            log.fatal("Problem: ", e);
        }
}

Memoryleaks in tomcat at Stop

1 Answers1