4

I am getting threadlocal memory leak errors in Tomcat and I am using ThreadPool, but have no implementation of ThreadLocal in my webapp.

SEVERE: The web application [/myWebApp] created a ThreadLocal with key of type [org.a pache.http.impl.cookie.DateUtils$DateFormatHolder$1] (value [org.apache.http.imp l.cookie.DateUtils$DateFormatHolder$1@4c2849]) and a value of type [java.lang.re f.SoftReference] (value [java.lang.ref.SoftReference@1e67280]) but failed to rem ove it when the web application was stopped. Threads are going to be renewed ove r time to try and avoid a probable memory leak.

What I dont understand is why i am getting threadlocal error although i have not implemented it? I want to get rid of these messages so I searched the web, and in here it is written that in order to clean the threadlocal i need to use:

ThreadLocal.remove()

but I have no implementation of ThreadLocal.. I'll be appreciated if someone show me a way.

Community
  • 1
  • 1
anvarik
  • 6,417
  • 5
  • 39
  • 53
  • You are probably creating Threads in your app. Have you tried destroying/cleaning them when the servlet is shutdown/restarted? It is usually not recommended to start Threads on your own in a JEE environment and rather use a framework like Quartz – Guillaume Polet May 02 '12 at 13:54
  • 1
    Yes, I am creating Threads but I am adding them to the pool. And after I finished, I shutdown the threadpoolexecutor. – anvarik May 02 '12 at 13:57

4 Answers4

2

The problem lies within your third party library. You cannot use thread locals in a thread pooled environment unless you really clean them up after the end of each request.

This article explains the problem: http://blog.maxant.co.uk/pebble/2008/09/23/1222200780000.html

jontro
  • 10,241
  • 6
  • 46
  • 71
  • I want to add this is not because of you are running your own thread pool, it's because of the third party is polluting the "native" thread pool that tomcat runs all incoming http requests on. – jontro May 02 '12 at 14:21
2

Clearly, something is creating that / those ThreadLocal instances. If it is not your code, then it must be some library you are using, or (unlikely) Tomcat itself.

I would start by looking at what might be creating instances of 

    org.apache.http.impl.cookie.DateUtils$DateFormatHolder$1

(That's an anonymous class in a nested class in DataUtils, by the way ... so unless something weird is coing on, the creation will be occuring in the DateUtils.java file.)

If examining the source code doesn't help, try debugging the Tomcat instance and setting a breakpoint on the ThreadLocal constructor(s).

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216
2

The ThreadLocal was obviously created by some framework or library you use (look which one is using HttpClient), but as you can see in the log the value is a SoftReference which should minimize the memory leak.

In fact you can see in the code for DateUtils that it is creating the Threadlocal...

pgras
  • 12,614
  • 4
  • 38
  • 46
  • 1
    Though the SoftReference would minimize the leak it won't resolve it. The ThreadLocal is tied to the classloader now, so hot deployments would result in OOM permagen. – John Vint May 02 '12 at 14:32
2

Here's the HttpClient JIRA: https://issues.apache.org/jira/browse/HTTPCLIENT-1216

From release 4.2.2 there's a clearThreadLocal() method, starting with 4.3 the cookie-DateUtils is deprecated and replaced with org.apache.http.client.utils.DateUtils.

Calling DateUtils.clearThreadLocal() once on shutdown is not enough, it only clears the ThreadLocal of the current thread, so you need to invoke it after performing an HTTP request which parses/formats dates, on that thread. This removes most of the performance benefit of using a ThreadLocal though.

Alternatively, if you perform HTTP requests from a thread under your control (not created by Tomcat), remember to shut down any thread pools/executors on application shutdown.

Big shame is that HttpClient could easily be modified to not override ThreadLocal, then the ThreadLocal wouldn't reference the webapp and its classloader, avoiding the bulk of the leak I believe :(

Stefan L
  • 1,529
  • 13
  • 20
  • Update: Supposedly fixed in httpclient 4.5.x after 4.5.1. Regarding avoiding overriding ThreadLocal, since Java 8 there's ThreadLocal.withInitial(Supplier) which however references your Supplier implementation, which in turn will pull in the webapp's classloader and "everything". – Stefan L Apr 24 '17 at 14:25